You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/07/01 04:51:20 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #46
See <https://builds.apache.org/job/beam_PerformanceTests_Python/46/display/redirect?page=changes>
Changes:
[lcwik] Properly convert milliseconds whether there's less than 3/more than 9
[klk] Add support for PipelineOptions parameters
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 893bf428a279e4c7f81accdd5e7eb7a4aaebb6b2 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1101340152487486225.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4778563135456701501.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson2371362200589201896.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson427103787516318281.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson131703439518488665.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-01 04:51:07,530 59220226 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/59220226/pkb.log
2017-07-01 04:51:07,530 59220226 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-01 04:51:07,530 59220226 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-01 04:51:07,861 59220226 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-01 04:51:07,953 59220226 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-01 04:51:07,956 59220226 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-01 04:51:07,956 59220226 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-01 04:51:13,798 59220226 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-01 04:51:13,800 59220226 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-01 04:51:13,853 59220226 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-01 04:51:13,854 59220226 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/59220226/pkb.log
2017-07-01 04:51:13,854 59220226 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/59220226/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Jenkins build is back to normal : beam_PerformanceTests_Python #67
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/67/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_Python #66
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/66/display/redirect?page=changes>
Changes:
[jbonofre] [maven-release-plugin] prepare branch release-2.1.0
[jbonofre] [maven-release-plugin] prepare for next development iteration
[jbonofre] Update Python SDK version
[tgroh] Website Mergebot Job
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f0549b464737d0d11ae1da8e0534622b10247208 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f0549b464737d0d11ae1da8e0534622b10247208
> git rev-list b8ac32641183b06a87517a169c8ecc5371996aed # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1315552242513050993.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3560950376733058620.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1400511056203541247.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5622132260084396564.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.2.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.2.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.2.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson2480493325630096163.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-05 18:00:18,994 61a23571 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/61a23571/pkb.log
2017-07-05 18:00:18,994 61a23571 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-05 18:00:18,995 61a23571 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-05 18:00:19,333 61a23571 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-05 18:00:19,468 61a23571 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-05 18:00:19,470 61a23571 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-05 18:00:19,471 61a23571 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-05 18:00:24,937 61a23571 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-05 18:00:24,939 61a23571 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-05 18:00:24,963 61a23571 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-05 18:00:24,963 61a23571 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/61a23571/pkb.log
2017-07-05 18:00:24,963 61a23571 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/61a23571/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #65
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/65/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b8ac32641183b06a87517a169c8ecc5371996aed (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b8ac32641183b06a87517a169c8ecc5371996aed
> git rev-list b8ac32641183b06a87517a169c8ecc5371996aed # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4240842984968854064.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1652381941545034549.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4489961629640165617.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1597072266746728941.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4272510997078060955.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-05 12:00:46,597 5a248716 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/5a248716/pkb.log
2017-07-05 12:00:46,597 5a248716 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-05 12:00:46,598 5a248716 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-05 12:00:46,830 5a248716 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-05 12:00:46,929 5a248716 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-05 12:00:46,931 5a248716 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-05 12:00:46,932 5a248716 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-05 12:00:51,646 5a248716 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-05 12:00:51,648 5a248716 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-05 12:00:51,671 5a248716 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-05 12:00:51,672 5a248716 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/5a248716/pkb.log
2017-07-05 12:00:51,672 5a248716 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/5a248716/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #64
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/64/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b8ac32641183b06a87517a169c8ecc5371996aed (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b8ac32641183b06a87517a169c8ecc5371996aed
> git rev-list b8ac32641183b06a87517a169c8ecc5371996aed # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5769114390110722930.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1748400145186379445.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3299410627385931806.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3028436834110752733.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6631889404632247339.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-05 06:00:12,035 2a2ea236 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/2a2ea236/pkb.log
2017-07-05 06:00:12,036 2a2ea236 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-05 06:00:12,036 2a2ea236 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-05 06:00:12,190 2a2ea236 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-05 06:00:12,285 2a2ea236 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-05 06:00:12,287 2a2ea236 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-05 06:00:12,288 2a2ea236 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-05 06:00:16,657 2a2ea236 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-05 06:00:16,659 2a2ea236 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-05 06:00:16,679 2a2ea236 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-05 06:00:16,680 2a2ea236 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/2a2ea236/pkb.log
2017-07-05 06:00:16,680 2a2ea236 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/2a2ea236/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #63
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/63/display/redirect?page=changes>
Changes:
[altay] For GCS operations use an http client with a default timeout value.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b8ac32641183b06a87517a169c8ecc5371996aed (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b8ac32641183b06a87517a169c8ecc5371996aed
> git rev-list 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1557826547367074992.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson9034803822663758452.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3364384303006842579.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson9079158364615018803.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6328705853689271627.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-05 00:00:12,235 41aba682 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/41aba682/pkb.log
2017-07-05 00:00:12,236 41aba682 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-05 00:00:12,236 41aba682 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-05 00:00:12,352 41aba682 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-05 00:00:12,444 41aba682 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-05 00:00:12,446 41aba682 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-05 00:00:12,446 41aba682 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-05 00:00:16,940 41aba682 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-05 00:00:16,943 41aba682 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-05 00:00:16,988 41aba682 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-05 00:00:16,989 41aba682 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/41aba682/pkb.log
2017-07-05 00:00:16,989 41aba682 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/41aba682/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #62
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/62/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243
> git rev-list 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3530178148755180937.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1691557569797211418.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4247422067582600197.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6992910845468077517.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8082688399571136665.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-04 18:00:14,745 531c460d MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/531c460d/pkb.log
2017-07-04 18:00:14,745 531c460d MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-04 18:00:14,746 531c460d MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-04 18:00:15,137 531c460d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-04 18:00:15,235 531c460d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-04 18:00:15,238 531c460d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-04 18:00:15,238 531c460d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-04 18:00:20,591 531c460d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-04 18:00:20,593 531c460d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-04 18:00:20,637 531c460d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-04 18:00:20,637 531c460d MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/531c460d/pkb.log
2017-07-04 18:00:20,637 531c460d MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/531c460d/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #61
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/61/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243
> git rev-list 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8491321205270513635.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7253917668948341361.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7889884933403546340.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3219280170612391401.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1550939016611445220.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-04 12:00:17,699 4fb13345 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/4fb13345/pkb.log
2017-07-04 12:00:17,699 4fb13345 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-04 12:00:17,700 4fb13345 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-04 12:00:17,941 4fb13345 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-04 12:00:18,033 4fb13345 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-04 12:00:18,036 4fb13345 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-04 12:00:18,036 4fb13345 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-04 12:00:22,742 4fb13345 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-04 12:00:22,744 4fb13345 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-04 12:00:22,781 4fb13345 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-04 12:00:22,782 4fb13345 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/4fb13345/pkb.log
2017-07-04 12:00:22,782 4fb13345 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/4fb13345/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #60
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/60/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243
> git rev-list 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8262255392103075408.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1638484120454865540.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3339565134672949632.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4542193965138184755.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1467594243710248187.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-04 06:00:09,919 ca432774 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ca432774/pkb.log
2017-07-04 06:00:09,919 ca432774 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-04 06:00:09,920 ca432774 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-04 06:00:10,030 ca432774 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-04 06:00:10,122 ca432774 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-04 06:00:10,124 ca432774 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-04 06:00:10,125 ca432774 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-04 06:00:14,451 ca432774 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-04 06:00:14,453 ca432774 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-04 06:00:14,466 ca432774 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-04 06:00:14,467 ca432774 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ca432774/pkb.log
2017-07-04 06:00:14,467 ca432774 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ca432774/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #59
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/59/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243
> git rev-list 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5319729041671723195.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson910036321182999356.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6678792084990695645.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8206565567529375711.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6027988165134881916.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-04 00:00:10,646 fcc6ae68 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/fcc6ae68/pkb.log
2017-07-04 00:00:10,646 fcc6ae68 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-04 00:00:10,647 fcc6ae68 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-04 00:00:10,881 fcc6ae68 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-04 00:00:10,974 fcc6ae68 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-04 00:00:10,976 fcc6ae68 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-04 00:00:10,977 fcc6ae68 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-04 00:00:15,293 fcc6ae68 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-04 00:00:15,295 fcc6ae68 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-04 00:00:15,318 fcc6ae68 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-04 00:00:15,318 fcc6ae68 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/fcc6ae68/pkb.log
2017-07-04 00:00:15,319 fcc6ae68 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/fcc6ae68/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #58
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/58/display/redirect?page=changes>
Changes:
[jbonofre] [BEAM-2530] Fix compilation of modules with Java 9 that depend on
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0bd47c07659909c6b3c2d04fbdb96cc30a6e0243
> git rev-list 11010cf08ec9ca90a843cc15ea8f0fd4910f7fab # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson398738363712742199.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6562891134953476816.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson705767665452622758.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6397254117961652724.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1599097198730595820.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-03 18:00:14,678 ec574d8a MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ec574d8a/pkb.log
2017-07-03 18:00:14,678 ec574d8a MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-03 18:00:14,679 ec574d8a MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-03 18:00:14,882 ec574d8a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-03 18:00:14,976 ec574d8a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-03 18:00:14,978 ec574d8a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-03 18:00:14,979 ec574d8a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-03 18:00:19,421 ec574d8a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-03 18:00:19,423 ec574d8a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-03 18:00:19,469 ec574d8a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-03 18:00:19,469 ec574d8a MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ec574d8a/pkb.log
2017-07-03 18:00:19,469 ec574d8a MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ec574d8a/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #57
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/57/display/redirect?page=changes>
Changes:
[jbonofre] Make modules that depend on Hadoop and Spark use the same version
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 11010cf08ec9ca90a843cc15ea8f0fd4910f7fab (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 11010cf08ec9ca90a843cc15ea8f0fd4910f7fab
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson848114572567495280.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5110391616487205278.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson2987471893144896789.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3200198662023629685.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6783585644846590227.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-03 12:01:13,124 018aed50 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/018aed50/pkb.log
2017-07-03 12:01:13,125 018aed50 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-03 12:01:13,125 018aed50 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-03 12:01:13,263 018aed50 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-03 12:01:13,347 018aed50 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-03 12:01:13,349 018aed50 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-03 12:01:13,350 018aed50 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-03 12:01:18,578 018aed50 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-03 12:01:18,580 018aed50 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-03 12:01:18,600 018aed50 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-03 12:01:18,600 018aed50 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/018aed50/pkb.log
2017-07-03 12:01:18,601 018aed50 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/018aed50/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #56
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/56/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4576748700706139736.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6903800885240862810.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5586061990740441910.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6962585333128719668.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson985593526633429529.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-03 06:00:14,777 b7321bf6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b7321bf6/pkb.log
2017-07-03 06:00:14,777 b7321bf6 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-03 06:00:14,778 b7321bf6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-03 06:00:15,180 b7321bf6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-03 06:00:15,285 b7321bf6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-03 06:00:15,288 b7321bf6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-03 06:00:15,288 b7321bf6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-03 06:00:19,614 b7321bf6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-03 06:00:19,616 b7321bf6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-03 06:00:19,629 b7321bf6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-03 06:00:19,630 b7321bf6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b7321bf6/pkb.log
2017-07-03 06:00:19,630 b7321bf6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b7321bf6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #55
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/55/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1078206742680113917.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson2316192936360848548.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7319961083691391935.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson237452952799757153.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7069244269548465320.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-03 00:00:12,367 f1c61e2c MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f1c61e2c/pkb.log
2017-07-03 00:00:12,367 f1c61e2c MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-03 00:00:12,368 f1c61e2c MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-03 00:00:12,479 f1c61e2c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-03 00:00:12,570 f1c61e2c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-03 00:00:12,573 f1c61e2c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-03 00:00:12,573 f1c61e2c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-03 00:00:16,811 f1c61e2c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-03 00:00:16,813 f1c61e2c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-03 00:00:16,863 f1c61e2c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-03 00:00:16,864 f1c61e2c MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f1c61e2c/pkb.log
2017-07-03 00:00:16,864 f1c61e2c MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f1c61e2c/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #54
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/54/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3492286056765877904.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4756539575917214415.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3465937549038740971.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1458180159627940942.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson132736262592478692.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-02 18:00:13,537 5033c00b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/5033c00b/pkb.log
2017-07-02 18:00:13,537 5033c00b MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-02 18:00:13,537 5033c00b MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-02 18:00:13,775 5033c00b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-02 18:00:13,872 5033c00b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-02 18:00:13,875 5033c00b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-02 18:00:13,875 5033c00b MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-02 18:00:18,570 5033c00b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-02 18:00:18,572 5033c00b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-02 18:00:18,614 5033c00b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-02 18:00:18,615 5033c00b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/5033c00b/pkb.log
2017-07-02 18:00:18,615 5033c00b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/5033c00b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #53
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/53/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8986720548569050875.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6372025829804883788.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7495457301498838725.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7597275457813855534.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1847315704528004084.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-02 12:00:10,448 2bd164cd MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/2bd164cd/pkb.log
2017-07-02 12:00:10,448 2bd164cd MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-02 12:00:10,448 2bd164cd MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-02 12:00:10,573 2bd164cd MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-02 12:00:10,668 2bd164cd MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-02 12:00:10,671 2bd164cd MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-02 12:00:10,671 2bd164cd MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-02 12:00:15,553 2bd164cd MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-02 12:00:15,555 2bd164cd MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-02 12:00:15,564 2bd164cd MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-02 12:00:15,564 2bd164cd MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/2bd164cd/pkb.log
2017-07-02 12:00:15,564 2bd164cd MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/2bd164cd/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #52
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/52/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson2827104935610194012.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4588075890827954969.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1722426357940558229.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1564900045185693019.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7799434107132124781.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-02 06:00:09,552 22aa279b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/22aa279b/pkb.log
2017-07-02 06:00:09,552 22aa279b MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-02 06:00:09,553 22aa279b MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-02 06:00:09,965 22aa279b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-02 06:00:10,055 22aa279b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-02 06:00:10,058 22aa279b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-02 06:00:10,058 22aa279b MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-02 06:00:14,510 22aa279b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-02 06:00:14,512 22aa279b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-02 06:00:14,551 22aa279b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-02 06:00:14,551 22aa279b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/22aa279b/pkb.log
2017-07-02 06:00:14,551 22aa279b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/22aa279b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #51
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/51/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7707668164309983300.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1819534988598542283.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3331762868621661269.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5157086837878786342.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8434170543108842354.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-02 00:15:48,724 5c1b33ad MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/5c1b33ad/pkb.log
2017-07-02 00:15:48,724 5c1b33ad MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-02 00:15:48,725 5c1b33ad MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-02 00:15:48,870 5c1b33ad MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-02 00:15:48,962 5c1b33ad MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-02 00:15:48,964 5c1b33ad MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-02 00:15:48,965 5c1b33ad MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-02 00:15:53,522 5c1b33ad MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-02 00:15:53,524 5c1b33ad MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-02 00:15:53,555 5c1b33ad MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-02 00:15:53,555 5c1b33ad MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/5c1b33ad/pkb.log
2017-07-02 00:15:53,556 5c1b33ad MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/5c1b33ad/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #50
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/50/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5314935011569793764.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1718130188280755496.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6554050934375921333.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4912086723058440201.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3159061272370643489.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-01 23:37:33,050 222a29c6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/222a29c6/pkb.log
2017-07-01 23:37:33,050 222a29c6 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-01 23:37:33,051 222a29c6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-01 23:37:33,261 222a29c6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-01 23:37:33,355 222a29c6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-01 23:37:33,357 222a29c6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-01 23:37:33,358 222a29c6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-01 23:37:37,758 222a29c6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-01 23:37:37,760 222a29c6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-01 23:37:37,801 222a29c6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-01 23:37:37,802 222a29c6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/222a29c6/pkb.log
2017-07-01 23:37:37,802 222a29c6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/222a29c6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #49
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/49/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3120695448153098172.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3231391480630912426.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1062087661805481249.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson3868619229622529282.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
Removing apache-beam 2.1.0.dev from easy-install.pth file
Adding apache-beam 2.1.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4917504516814239032.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-01 22:58:19,736 c95b4d30 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/c95b4d30/pkb.log
2017-07-01 22:58:19,736 c95b4d30 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-01 22:58:19,736 c95b4d30 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-01 22:58:20,071 c95b4d30 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-01 22:58:20,161 c95b4d30 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-01 22:58:20,163 c95b4d30 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-01 22:58:20,164 c95b4d30 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-01 22:58:24,642 c95b4d30 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-01 22:58:24,644 c95b4d30 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-01 22:58:24,656 c95b4d30 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-01 22:58:24,657 c95b4d30 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/c95b4d30/pkb.log
2017-07-01 22:58:24,657 c95b4d30 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/c95b4d30/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Build failed in Jenkins: beam_PerformanceTests_Python #48
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/48/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson6529342162802099299.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson4077271160419505959.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson318531803745068542.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson868776907272104005.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson5135332014924134341.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-01 10:37:22,799 dddb33d4 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/dddb33d4/pkb.log
2017-07-01 10:37:22,799 dddb33d4 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-01 10:37:22,800 dddb33d4 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-01 10:37:23,028 dddb33d4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-01 10:37:23,122 dddb33d4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-01 10:37:23,125 dddb33d4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-01 10:37:23,125 dddb33d4 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-01 10:37:27,519 dddb33d4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-01 10:37:27,522 dddb33d4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-01 10:37:27,571 dddb33d4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-01 10:37:27,572 dddb33d4 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/dddb33d4/pkb.log
2017-07-01 10:37:27,572 dddb33d4 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/dddb33d4/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com
Re: Build failed in Jenkins: beam_PerformanceTests_Python #47
Posted by Ted Yu <yu...@gmail.com>.
I checked out PerfKitBenchmarker.
It seems we can get rid of the error either by defining a default value for
runner_profile_override :
+def AddRunnerProfileMvnArgument(service_type, mvn_command,
+ runner_profile_override):
Or passing None in the places where AddRunnerProfileMvnArgument is called.
On Sat, Jul 1, 2017 at 9:19 AM, Ted Yu <yu...@gmail.com> wrote:
> > File "<https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchma
> rk_helper.py",>
> > line 136, in InitializeBeamRepo
> > mvn_command)
> > TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2
> > given)
>
> Looks like 3rd parameter is needed to initialize repo.
>
> beam_benchmark_helper.py doesn't seem to be in source repo, though.
>
>
> On Fri, Jun 30, 2017 at 10:55 PM, Kenneth Knowles <kl...@google.com.invalid>
> wrote:
>
>> This build has been failing for two days. Does anyone have any insights?
>>
>> On Fri, Jun 30, 2017 at 10:50 PM, Apache Jenkins Server <
>> jenkins@builds.apache.org> wrote:
>>
>> > See <https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/47/display/redirect>
>> >
>> > ------------------------------------------
>> > Started by timer
>> > [EnvInject] - Loading node environment variables.
>> > Building remotely on beam3 (beam) in workspace <
>> https://builds.apache.org/
>> > job/beam_PerformanceTests_Python/ws/>
>> > > git rev-parse --is-inside-work-tree # timeout=10
>> > Fetching changes from the remote Git repository
>> > > git config remote.origin.url https://github.com/apache/beam.git #
>> > timeout=10
>> > Fetching upstream changes from https://github.com/apache/beam.git
>> > > git --version # timeout=10
>> > > git fetch --tags --progress https://github.com/apache/beam.git
>> > +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:
>> > refs/remotes/origin/pr/${ghprbPullId}/*
>> > > git rev-parse origin/master^{commit} # timeout=10
>> > Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720
>> > (origin/master)
>> > > git config core.sparsecheckout # timeout=10
>> > > git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
>> > > git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
>> > Cleaning workspace
>> > > git rev-parse --verify HEAD # timeout=10
>> > Resetting working tree
>> > > git reset --hard # timeout=10
>> > > git clean -fdx # timeout=10
>> > [EnvInject] - Executing scripts and injecting environment variables
>> after
>> > the SCM step.
>> > [EnvInject] - Injecting as environment variables the properties content
>> > SPARK_LOCAL_IP=127.0.0.1
>> >
>> > [EnvInject] - Variables injected successfully.
>> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
>> > hudson1915973589795310141.sh
>> > + rm -rf PerfKitBenchmarker
>> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
>> > hudson7830718553943537745.sh
>> > + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.
>> git
>> > Cloning into 'PerfKitBenchmarker'...
>> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
>> > hudson7887504202385599721.sh
>> > + pip install --user -r PerfKitBenchmarker/requirements.txt
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > python-gflags==3.1.1 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from -r PerfKitBenchmarker/requirements.txt (line 14))
>> > Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in
>> > /usr/local/lib/python2.7/dist-packages (from -r
>> PerfKitBenchmarker/requirements.txt
>> > (line 15))
>> > Requirement already satisfied (use --upgrade to upgrade): setuptools in
>> > /usr/lib/python2.7/dist-packages (from -r
>> PerfKitBenchmarker/requirements.txt
>> > (line 16))
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from -r PerfKitBenchmarker/requirements.txt (line 17))
>> > Installing extra requirements: 'windows'
>> > Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3
>> in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
>> > PerfKitBenchmarker/requirements.txt (line 18))
>> > Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3
>> > in /home/jenkins/.local/lib/python2.7/site-packages (from -r
>> > PerfKitBenchmarker/requirements.txt (line 19))
>> > Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12
>> in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
>> > PerfKitBenchmarker/requirements.txt (line 20))
>> > Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
>> > PerfKitBenchmarker/requirements.txt (line 21))
>> > Requirement already satisfied (use --upgrade to upgrade): numpy in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
>> > PerfKitBenchmarker/requirements.txt (line 22))
>> > Requirement already satisfied (use --upgrade to upgrade): functools32 in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
>> > PerfKitBenchmarker/requirements.txt (line 23))
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages
>> > (from -r PerfKitBenchmarker/requirements.txt (line 24))
>> > Cleaning up...
>> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
>> > hudson8072560625143652403.sh
>> > + pip install --user -e 'sdks/python/[gcp,test]'
>> > Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/sdks/python>
>> > Running setup.py (path:<https://builds.apache.
>> > org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info
>> > for package from file://<https://builds.apache.
>> > org/job/beam_PerformanceTests_Python/ws/sdks/python>
>> > <https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version
>> > 1.5.4 of pip. However, version 7.0.0 is recommended.
>> > _PIP_VERSION, REQUIRED_PIP_VERSION
>> > no previously-included directories found matching 'doc/.build'
>> >
>> > Installed <https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
>> >
>> > warning: no files found matching 'README.md'
>> > warning: no files found matching 'NOTICE'
>> > warning: no files found matching 'LICENSE'
>> > Installing extra requirements: 'gcp,test'
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> crcmod>=1.7,<2.0
>> > in /home/jenkins/.local/lib/python2.7/site-packages (from
>> > apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from
>> > apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> grpcio>=1.0,<2.0
>> > in /home/jenkins/.local/lib/python2.7/site-packages (from
>> > apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from
>> > apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from
>> > apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > googledatastore==7.0.1 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0
>> > in /home/jenkins/.local/lib/python2.7/site-packages (from
>> > grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4
>> in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from
>> > grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7
>> in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from
>> > oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/pytho
>> n2.7/site-packages
>> > (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in
>> > /home/jenkins/.local/lib/python2.7/site-packages (from
>> > oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade): setuptools in
>> > /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->
>> > apache-beam==2.1.0.dev)
>> > Requirement already satisfied (use --upgrade to upgrade):
>> > googleapis-common-protos>=1.5.2,<2.0dev in
>> /home/jenkins/.local/lib/python2.7/site-packages
>> > (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-
>> > beam==2.1.0.dev)
>> > Installing collected packages: apache-beam
>> > Running setup.py develop for apache-beam
>> > <https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version
>> > 1.5.4 of pip. However, version 7.0.0 is recommended.
>> > _PIP_VERSION, REQUIRED_PIP_VERSION
>> >
>> > warning: no files found matching 'README.md'
>> > warning: no files found matching 'NOTICE'
>> > warning: no files found matching 'LICENSE'
>> > Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-
>> beam.egg-link
>> > (link to .)
>> > apache-beam 2.1.0.dev is already the active version in
>> easy-install.pth
>> >
>> > Installed <https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/sdks/python>
>> > Successfully installed apache-beam
>> > Cleaning up...
>> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
>> > hudson7511664461243177645.sh
>> > + python PerfKitBenchmarker/pkb.py --project=apache-beam-testing
>> > --dpb_log_level=INFO --maven_binary=/home/jenkins/t
>> ools/maven/latest/bin/mvn
>> > --bigquery_table=beam_performance.pkb_results --official=true
>> > --beam_sdk=python --benchmarks=beam_integration_benchmark
>> > --beam_it_args=--project=apache-beam-testing,--staging_
>> > location=gs://temp-storage-for-end-to-end-tests/staging-
>> > it,--temp_location=gs://temp-storage-for-end-to-end-tests/
>> > temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-
>> it-cloud/output
>> > WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is
>> not
>> > a directory.
>> > 2017-07-01 05:50:07,346 10b71d64 MainThread INFO Verbose logging to:
>> > /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
>> > 2017-07-01 05:50:07,347 10b71d64 MainThread INFO PerfKitBenchmarker
>> > version: v1.12.0-53-ga07b524
>> > 2017-07-01 05:50:07,347 10b71d64 MainThread INFO Flag values:
>> > --beam_sdk=python
>> > --beam_it_args=--project=apache-beam-testing,--staging_
>> > location=gs://temp-storage-for-end-to-end-tests/staging-
>> > it,--temp_location=gs://temp-storage-for-end-to-end-tests/
>> > temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-
>> it-cloud/output
>> > --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
>> > --project=apache-beam-testing
>> > --bigquery_table=beam_performance.pkb_results
>> > --official
>> > --dpb_log_level=INFO
>> > --benchmarks=beam_integration_benchmark
>> > 2017-07-01 05:50:07,785 10b71d64 MainThread WARNING The key "flags" was
>> > not in the default config, but was in user overrides. This may indicate
>> a
>> > typo.
>> > 2017-07-01 05:50:08,046 10b71d64 MainThread
>> beam_integration_benchmark(1/1)
>> > INFO Provisioning resources for benchmark beam_integration_benchmark
>> > 2017-07-01 05:50:08,049 10b71d64 MainThread
>> beam_integration_benchmark(1/1)
>> > INFO Preparing benchmark beam_integration_benchmark
>> > 2017-07-01 05:50:08,049 10b71d64 MainThread
>> beam_integration_benchmark(1/1)
>> > INFO Running: git clone https://github.com/apache/beam.git
>> > 2017-07-01 05:50:13,212 10b71d64 MainThread
>> beam_integration_benchmark(1/1)
>> > ERROR Error during benchmark beam_integration_benchmark
>> > Traceback (most recent call last):
>> > File "<https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in
>> > RunBenchmark
>> > DoPreparePhase(spec, detailed_timer)
>> > File "<https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in
>> > DoPreparePhase
>> > spec.BenchmarkPrepare(spec)
>> > File "<https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_
>> > benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
>> > beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
>> > File "<https://builds.apache.org/job/beam_PerformanceTests_
>> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchma
>> rk_helper.py",>
>> > line 136, in InitializeBeamRepo
>> > mvn_command)
>> > TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2
>> > given)
>> > 2017-07-01 05:50:13,214 10b71d64 MainThread
>> beam_integration_benchmark(1/1)
>> > ERROR Benchmark 1/1 beam_integration_benchmark (UID:
>> > beam_integration_benchmark0) failed. Execution will continue.
>> > 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Benchmark run
>> > statuses:
>> > ---------------------------------------------------------------
>> > Name UID Status
>> > ---------------------------------------------------------------
>> > beam_integration_benchmark beam_integration_benchmark0 FAILED
>> > ---------------------------------------------------------------
>> > Success rate: 0.00% (0/1)
>> > 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Complete logs can
>> be
>> > found at: /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
>> > 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Completion statuses
>> > can be found at: /tmp/perfkitbenchmarker/runs/
>> > 10b71d64/completion_statuses.json
>> > Build step 'Execute shell' marked build as failure
>> > Not sending mail to unregistered user klk@google.com
>> > Not sending mail to unregistered user sisk@google.com
>> > Not sending mail to unregistered user mairbek@google.com
>> >
>>
>
>
Re: Build failed in Jenkins: beam_PerformanceTests_Python #47
Posted by Ted Yu <yu...@gmail.com>.
> File "<https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_
benchmark_helper.py",>
> line 136, in InitializeBeamRepo
> mvn_command)
> TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2
> given)
Looks like 3rd parameter is needed to initialize repo.
beam_benchmark_helper.py doesn't seem to be in source repo, though.
On Fri, Jun 30, 2017 at 10:55 PM, Kenneth Knowles <kl...@google.com.invalid>
wrote:
> This build has been failing for two days. Does anyone have any insights?
>
> On Fri, Jun 30, 2017 at 10:50 PM, Apache Jenkins Server <
> jenkins@builds.apache.org> wrote:
>
> > See <https://builds.apache.org/job/beam_PerformanceTests_
> > Python/47/display/redirect>
> >
> > ------------------------------------------
> > Started by timer
> > [EnvInject] - Loading node environment variables.
> > Building remotely on beam3 (beam) in workspace <
> https://builds.apache.org/
> > job/beam_PerformanceTests_Python/ws/>
> > > git rev-parse --is-inside-work-tree # timeout=10
> > Fetching changes from the remote Git repository
> > > git config remote.origin.url https://github.com/apache/beam.git #
> > timeout=10
> > Fetching upstream changes from https://github.com/apache/beam.git
> > > git --version # timeout=10
> > > git fetch --tags --progress https://github.com/apache/beam.git
> > +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:
> > refs/remotes/origin/pr/${ghprbPullId}/*
> > > git rev-parse origin/master^{commit} # timeout=10
> > Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720
> > (origin/master)
> > > git config core.sparsecheckout # timeout=10
> > > git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> > > git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
> > Cleaning workspace
> > > git rev-parse --verify HEAD # timeout=10
> > Resetting working tree
> > > git reset --hard # timeout=10
> > > git clean -fdx # timeout=10
> > [EnvInject] - Executing scripts and injecting environment variables after
> > the SCM step.
> > [EnvInject] - Injecting as environment variables the properties content
> > SPARK_LOCAL_IP=127.0.0.1
> >
> > [EnvInject] - Variables injected successfully.
> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> > hudson1915973589795310141.sh
> > + rm -rf PerfKitBenchmarker
> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> > hudson7830718553943537745.sh
> > + git clone https://github.com/GoogleCloudPlatform/
> PerfKitBenchmarker.git
> > Cloning into 'PerfKitBenchmarker'...
> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> > hudson7887504202385599721.sh
> > + pip install --user -r PerfKitBenchmarker/requirements.txt
> > Requirement already satisfied (use --upgrade to upgrade):
> > python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from -r PerfKitBenchmarker/requirements.txt (line 14))
> > Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in
> > /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/
> requirements.txt
> > (line 15))
> > Requirement already satisfied (use --upgrade to upgrade): setuptools in
> > /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/
> requirements.txt
> > (line 16))
> > Requirement already satisfied (use --upgrade to upgrade):
> > colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from -r PerfKitBenchmarker/requirements.txt (line 17))
> > Installing extra requirements: 'windows'
> > Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
> > PerfKitBenchmarker/requirements.txt (line 18))
> > Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3
> > in /home/jenkins/.local/lib/python2.7/site-packages (from -r
> > PerfKitBenchmarker/requirements.txt (line 19))
> > Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
> > PerfKitBenchmarker/requirements.txt (line 20))
> > Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
> > PerfKitBenchmarker/requirements.txt (line 21))
> > Requirement already satisfied (use --upgrade to upgrade): numpy in
> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
> > PerfKitBenchmarker/requirements.txt (line 22))
> > Requirement already satisfied (use --upgrade to upgrade): functools32 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from -r
> > PerfKitBenchmarker/requirements.txt (line 23))
> > Requirement already satisfied (use --upgrade to upgrade):
> > contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from -r PerfKitBenchmarker/requirements.txt (line 24))
> > Cleaning up...
> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> > hudson8072560625143652403.sh
> > + pip install --user -e 'sdks/python/[gcp,test]'
> > Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/sdks/python>
> > Running setup.py (path:<https://builds.apache.
> > org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info
> > for package from file://<https://builds.apache.
> > org/job/beam_PerformanceTests_Python/ws/sdks/python>
> > <https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version
> > 1.5.4 of pip. However, version 7.0.0 is recommended.
> > _PIP_VERSION, REQUIRED_PIP_VERSION
> > no previously-included directories found matching 'doc/.build'
> >
> > Installed <https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
> >
> > warning: no files found matching 'README.md'
> > warning: no files found matching 'NOTICE'
> > warning: no files found matching 'LICENSE'
> > Installing extra requirements: 'gcp,test'
> > Requirement already satisfied (use --upgrade to upgrade):
> > avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> crcmod>=1.7,<2.0
> > in /home/jenkins/.local/lib/python2.7/site-packages (from
> > apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from
> > apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> grpcio>=1.0,<2.0
> > in /home/jenkins/.local/lib/python2.7/site-packages (from
> > apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from
> > apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from
> > apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > googledatastore==7.0.1 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0
> > in /home/jenkins/.local/lib/python2.7/site-packages (from
> > grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4
> in
> > /home/jenkins/.local/lib/python2.7/site-packages (from
> > grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7
> in
> > /home/jenkins/.local/lib/python2.7/site-packages (from
> > oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in
> > /home/jenkins/.local/lib/python2.7/site-packages (from
> > oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade): setuptools in
> > /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->
> > apache-beam==2.1.0.dev)
> > Requirement already satisfied (use --upgrade to upgrade):
> > googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/
> python2.7/site-packages
> > (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-
> > beam==2.1.0.dev)
> > Installing collected packages: apache-beam
> > Running setup.py develop for apache-beam
> > <https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version
> > 1.5.4 of pip. However, version 7.0.0 is recommended.
> > _PIP_VERSION, REQUIRED_PIP_VERSION
> >
> > warning: no files found matching 'README.md'
> > warning: no files found matching 'NOTICE'
> > warning: no files found matching 'LICENSE'
> > Creating /home/jenkins/.local/lib/python2.7/site-packages/
> apache-beam.egg-link
> > (link to .)
> > apache-beam 2.1.0.dev is already the active version in
> easy-install.pth
> >
> > Installed <https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/sdks/python>
> > Successfully installed apache-beam
> > Cleaning up...
> > [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> > hudson7511664461243177645.sh
> > + python PerfKitBenchmarker/pkb.py --project=apache-beam-testing
> > --dpb_log_level=INFO --maven_binary=/home/jenkins/
> tools/maven/latest/bin/mvn
> > --bigquery_table=beam_performance.pkb_results --official=true
> > --beam_sdk=python --benchmarks=beam_integration_benchmark
> > --beam_it_args=--project=apache-beam-testing,--staging_
> > location=gs://temp-storage-for-end-to-end-tests/staging-
> > it,--temp_location=gs://temp-storage-for-end-to-end-tests/
> > temp-it,--output=gs://temp-storage-for-end-to-end-tests/
> py-it-cloud/output
> > WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is
> not
> > a directory.
> > 2017-07-01 05:50:07,346 10b71d64 MainThread INFO Verbose logging to:
> > /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
> > 2017-07-01 05:50:07,347 10b71d64 MainThread INFO PerfKitBenchmarker
> > version: v1.12.0-53-ga07b524
> > 2017-07-01 05:50:07,347 10b71d64 MainThread INFO Flag values:
> > --beam_sdk=python
> > --beam_it_args=--project=apache-beam-testing,--staging_
> > location=gs://temp-storage-for-end-to-end-tests/staging-
> > it,--temp_location=gs://temp-storage-for-end-to-end-tests/
> > temp-it,--output=gs://temp-storage-for-end-to-end-tests/
> py-it-cloud/output
> > --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
> > --project=apache-beam-testing
> > --bigquery_table=beam_performance.pkb_results
> > --official
> > --dpb_log_level=INFO
> > --benchmarks=beam_integration_benchmark
> > 2017-07-01 05:50:07,785 10b71d64 MainThread WARNING The key "flags" was
> > not in the default config, but was in user overrides. This may indicate a
> > typo.
> > 2017-07-01 05:50:08,046 10b71d64 MainThread beam_integration_benchmark(1/
> 1)
> > INFO Provisioning resources for benchmark beam_integration_benchmark
> > 2017-07-01 05:50:08,049 10b71d64 MainThread beam_integration_benchmark(1/
> 1)
> > INFO Preparing benchmark beam_integration_benchmark
> > 2017-07-01 05:50:08,049 10b71d64 MainThread beam_integration_benchmark(1/
> 1)
> > INFO Running: git clone https://github.com/apache/beam.git
> > 2017-07-01 05:50:13,212 10b71d64 MainThread beam_integration_benchmark(1/
> 1)
> > ERROR Error during benchmark beam_integration_benchmark
> > Traceback (most recent call last):
> > File "<https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in
> > RunBenchmark
> > DoPreparePhase(spec, detailed_timer)
> > File "<https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in
> > DoPreparePhase
> > spec.BenchmarkPrepare(spec)
> > File "<https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_
> > benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
> > beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
> > File "<https://builds.apache.org/job/beam_PerformanceTests_
> > Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_
> benchmark_helper.py",>
> > line 136, in InitializeBeamRepo
> > mvn_command)
> > TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2
> > given)
> > 2017-07-01 05:50:13,214 10b71d64 MainThread beam_integration_benchmark(1/
> 1)
> > ERROR Benchmark 1/1 beam_integration_benchmark (UID:
> > beam_integration_benchmark0) failed. Execution will continue.
> > 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Benchmark run
> > statuses:
> > ---------------------------------------------------------------
> > Name UID Status
> > ---------------------------------------------------------------
> > beam_integration_benchmark beam_integration_benchmark0 FAILED
> > ---------------------------------------------------------------
> > Success rate: 0.00% (0/1)
> > 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Complete logs can be
> > found at: /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
> > 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Completion statuses
> > can be found at: /tmp/perfkitbenchmarker/runs/
> > 10b71d64/completion_statuses.json
> > Build step 'Execute shell' marked build as failure
> > Not sending mail to unregistered user klk@google.com
> > Not sending mail to unregistered user sisk@google.com
> > Not sending mail to unregistered user mairbek@google.com
> >
>
Re: Build failed in Jenkins: beam_PerformanceTests_Python #47
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
This build has been failing for two days. Does anyone have any insights?
On Fri, Jun 30, 2017 at 10:50 PM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:
> See <https://builds.apache.org/job/beam_PerformanceTests_
> Python/47/display/redirect>
>
> ------------------------------------------
> Started by timer
> [EnvInject] - Loading node environment variables.
> Building remotely on beam3 (beam) in workspace <https://builds.apache.org/
> job/beam_PerformanceTests_Python/ws/>
> > git rev-parse --is-inside-work-tree # timeout=10
> Fetching changes from the remote Git repository
> > git config remote.origin.url https://github.com/apache/beam.git #
> timeout=10
> Fetching upstream changes from https://github.com/apache/beam.git
> > git --version # timeout=10
> > git fetch --tags --progress https://github.com/apache/beam.git
> +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:
> refs/remotes/origin/pr/${ghprbPullId}/*
> > git rev-parse origin/master^{commit} # timeout=10
> Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720
> (origin/master)
> > git config core.sparsecheckout # timeout=10
> > git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> > git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
> Cleaning workspace
> > git rev-parse --verify HEAD # timeout=10
> Resetting working tree
> > git reset --hard # timeout=10
> > git clean -fdx # timeout=10
> [EnvInject] - Executing scripts and injecting environment variables after
> the SCM step.
> [EnvInject] - Injecting as environment variables the properties content
> SPARK_LOCAL_IP=127.0.0.1
>
> [EnvInject] - Variables injected successfully.
> [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> hudson1915973589795310141.sh
> + rm -rf PerfKitBenchmarker
> [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> hudson7830718553943537745.sh
> + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
> Cloning into 'PerfKitBenchmarker'...
> [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> hudson7887504202385599721.sh
> + pip install --user -r PerfKitBenchmarker/requirements.txt
> Requirement already satisfied (use --upgrade to upgrade):
> python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages
> (from -r PerfKitBenchmarker/requirements.txt (line 14))
> Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in
> /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt
> (line 15))
> Requirement already satisfied (use --upgrade to upgrade): setuptools in
> /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt
> (line 16))
> Requirement already satisfied (use --upgrade to upgrade):
> colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from -r PerfKitBenchmarker/requirements.txt (line 17))
> Installing extra requirements: 'windows'
> Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in
> /home/jenkins/.local/lib/python2.7/site-packages (from -r
> PerfKitBenchmarker/requirements.txt (line 18))
> Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3
> in /home/jenkins/.local/lib/python2.7/site-packages (from -r
> PerfKitBenchmarker/requirements.txt (line 19))
> Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in
> /home/jenkins/.local/lib/python2.7/site-packages (from -r
> PerfKitBenchmarker/requirements.txt (line 20))
> Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in
> /home/jenkins/.local/lib/python2.7/site-packages (from -r
> PerfKitBenchmarker/requirements.txt (line 21))
> Requirement already satisfied (use --upgrade to upgrade): numpy in
> /home/jenkins/.local/lib/python2.7/site-packages (from -r
> PerfKitBenchmarker/requirements.txt (line 22))
> Requirement already satisfied (use --upgrade to upgrade): functools32 in
> /home/jenkins/.local/lib/python2.7/site-packages (from -r
> PerfKitBenchmarker/requirements.txt (line 23))
> Requirement already satisfied (use --upgrade to upgrade):
> contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages
> (from -r PerfKitBenchmarker/requirements.txt (line 24))
> Cleaning up...
> [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> hudson8072560625143652403.sh
> + pip install --user -e 'sdks/python/[gcp,test]'
> Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/sdks/python>
> Running setup.py (path:<https://builds.apache.
> org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info
> for package from file://<https://builds.apache.
> org/job/beam_PerformanceTests_Python/ws/sdks/python>
> <https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version
> 1.5.4 of pip. However, version 7.0.0 is recommended.
> _PIP_VERSION, REQUIRED_PIP_VERSION
> no previously-included directories found matching 'doc/.build'
>
> Installed <https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
>
> warning: no files found matching 'README.md'
> warning: no files found matching 'NOTICE'
> warning: no files found matching 'LICENSE'
> Installing extra requirements: 'gcp,test'
> Requirement already satisfied (use --upgrade to upgrade):
> avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0
> in /home/jenkins/.local/lib/python2.7/site-packages (from
> apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in
> /home/jenkins/.local/lib/python2.7/site-packages (from
> apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0
> in /home/jenkins/.local/lib/python2.7/site-packages (from
> apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in
> /home/jenkins/.local/lib/python2.7/site-packages (from
> apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in
> /home/jenkins/.local/lib/python2.7/site-packages (from
> apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages
> (from apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0
> in /home/jenkins/.local/lib/python2.7/site-packages (from
> grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in
> /home/jenkins/.local/lib/python2.7/site-packages (from
> grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in
> /home/jenkins/.local/lib/python2.7/site-packages (from
> oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages
> (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in
> /home/jenkins/.local/lib/python2.7/site-packages (from
> oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade): setuptools in
> /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->
> apache-beam==2.1.0.dev)
> Requirement already satisfied (use --upgrade to upgrade):
> googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages
> (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-
> beam==2.1.0.dev)
> Installing collected packages: apache-beam
> Running setup.py develop for apache-beam
> <https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version
> 1.5.4 of pip. However, version 7.0.0 is recommended.
> _PIP_VERSION, REQUIRED_PIP_VERSION
>
> warning: no files found matching 'README.md'
> warning: no files found matching 'NOTICE'
> warning: no files found matching 'LICENSE'
> Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link
> (link to .)
> apache-beam 2.1.0.dev is already the active version in easy-install.pth
>
> Installed <https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/sdks/python>
> Successfully installed apache-beam
> Cleaning up...
> [beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/
> hudson7511664461243177645.sh
> + python PerfKitBenchmarker/pkb.py --project=apache-beam-testing
> --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
> --bigquery_table=beam_performance.pkb_results --official=true
> --beam_sdk=python --benchmarks=beam_integration_benchmark
> --beam_it_args=--project=apache-beam-testing,--staging_
> location=gs://temp-storage-for-end-to-end-tests/staging-
> it,--temp_location=gs://temp-storage-for-end-to-end-tests/
> temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
> WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not
> a directory.
> 2017-07-01 05:50:07,346 10b71d64 MainThread INFO Verbose logging to:
> /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
> 2017-07-01 05:50:07,347 10b71d64 MainThread INFO PerfKitBenchmarker
> version: v1.12.0-53-ga07b524
> 2017-07-01 05:50:07,347 10b71d64 MainThread INFO Flag values:
> --beam_sdk=python
> --beam_it_args=--project=apache-beam-testing,--staging_
> location=gs://temp-storage-for-end-to-end-tests/staging-
> it,--temp_location=gs://temp-storage-for-end-to-end-tests/
> temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
> --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
> --project=apache-beam-testing
> --bigquery_table=beam_performance.pkb_results
> --official
> --dpb_log_level=INFO
> --benchmarks=beam_integration_benchmark
> 2017-07-01 05:50:07,785 10b71d64 MainThread WARNING The key "flags" was
> not in the default config, but was in user overrides. This may indicate a
> typo.
> 2017-07-01 05:50:08,046 10b71d64 MainThread beam_integration_benchmark(1/1)
> INFO Provisioning resources for benchmark beam_integration_benchmark
> 2017-07-01 05:50:08,049 10b71d64 MainThread beam_integration_benchmark(1/1)
> INFO Preparing benchmark beam_integration_benchmark
> 2017-07-01 05:50:08,049 10b71d64 MainThread beam_integration_benchmark(1/1)
> INFO Running: git clone https://github.com/apache/beam.git
> 2017-07-01 05:50:13,212 10b71d64 MainThread beam_integration_benchmark(1/1)
> ERROR Error during benchmark beam_integration_benchmark
> Traceback (most recent call last):
> File "<https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in
> RunBenchmark
> DoPreparePhase(spec, detailed_timer)
> File "<https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in
> DoPreparePhase
> spec.BenchmarkPrepare(spec)
> File "<https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_
> benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
> beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
> File "<https://builds.apache.org/job/beam_PerformanceTests_
> Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",>
> line 136, in InitializeBeamRepo
> mvn_command)
> TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2
> given)
> 2017-07-01 05:50:13,214 10b71d64 MainThread beam_integration_benchmark(1/1)
> ERROR Benchmark 1/1 beam_integration_benchmark (UID:
> beam_integration_benchmark0) failed. Execution will continue.
> 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Benchmark run
> statuses:
> ---------------------------------------------------------------
> Name UID Status
> ---------------------------------------------------------------
> beam_integration_benchmark beam_integration_benchmark0 FAILED
> ---------------------------------------------------------------
> Success rate: 0.00% (0/1)
> 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Complete logs can be
> found at: /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
> 2017-07-01 05:50:13,237 10b71d64 MainThread INFO Completion statuses
> can be found at: /tmp/perfkitbenchmarker/runs/
> 10b71d64/completion_statuses.json
> Build step 'Execute shell' marked build as failure
> Not sending mail to unregistered user klk@google.com
> Not sending mail to unregistered user sisk@google.com
> Not sending mail to unregistered user mairbek@google.com
>
Build failed in Jenkins: beam_PerformanceTests_Python #47
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/47/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0e429b33ff85eba08da5018c9febd0b99b44f720 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e429b33ff85eba08da5018c9febd0b99b44f720
> git rev-list 0e429b33ff85eba08da5018c9febd0b99b44f720 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson1915973589795310141.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7830718553943537745.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7887504202385599721.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): python-gflags==3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson8072560625143652403.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/nose-1.3.7-py2.7.egg>
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.2.0,<=3.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.10,<=0.5.11 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery>=0.23.0,<0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<4.0.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/lib/python2.7/dist-packages (from protobuf>=3.2.0,<=3.3.0->apache-beam==2.1.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.1.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating /home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link (link to .)
apache-beam 2.1.0.dev is already the active version in easy-install.pth
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/hudson7511664461243177645.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
WARNING:root:File resource loader root perfkitbenchmarker/data/ycsb is not a directory.
2017-07-01 05:50:07,346 10b71d64 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
2017-07-01 05:50:07,347 10b71d64 MainThread INFO PerfKitBenchmarker version: v1.12.0-53-ga07b524
2017-07-01 05:50:07,347 10b71d64 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-07-01 05:50:07,785 10b71d64 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-07-01 05:50:08,046 10b71d64 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-07-01 05:50:08,049 10b71d64 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-07-01 05:50:08,049 10b71d64 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-07-01 05:50:13,212 10b71d64 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 555, in RunBenchmark
DoPreparePhase(spec, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 450, in DoPreparePhase
spec.BenchmarkPrepare(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 91, in Prepare
beam_benchmark_helper.InitializeBeamRepo(benchmark_spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 136, in InitializeBeamRepo
mvn_command)
TypeError: AddRunnerProfileMvnArgument() takes exactly 3 arguments (2 given)
2017-07-01 05:50:13,214 10b71d64 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-07-01 05:50:13,237 10b71d64 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-07-01 05:50:13,237 10b71d64 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/10b71d64/pkb.log
2017-07-01 05:50:13,237 10b71d64 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/10b71d64/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sisk@google.com
Not sending mail to unregistered user mairbek@google.com