You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/11/11 06:16:03 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #546
See <https://builds.apache.org/job/beam_PerformanceTests_Python/546/display/redirect?page=changes>
Changes:
[chamikara] [BEAM-3060] Adds TextIOIT for DirectRunner and local filesystem
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list 51c938d4e32dcc3b2c6b4e847f481e3e00d5e4b0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1692132497381291067.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7651053548935590924.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4033677284270526395.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3140794683268532976.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1019963290388668867.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-11 06:00:26,703 21269b82 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/21269b82/pkb.log
2017-11-11 06:00:26,706 21269b82 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-11 06:00:26,707 21269b82 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-11 06:00:27,057 21269b82 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-11 06:00:27,058 21269b82 MainThread INFO Initializing the edw service decoder
2017-11-11 06:00:27,286 21269b82 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-11 06:00:27,297 21269b82 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-11 06:00:27,298 21269b82 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-11 06:00:39,603 21269b82 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-11 06:15:41,353 21269b82 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-11 06:15:41,375 21269b82 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/21269b82/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-11 06:15:41,644 21269b82 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/21269b82/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/21269b82/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-11 06:15:41,644 21269b82 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-11 06:15:41,645 21269b82 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-11 06:15:41,647 21269b82 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-11 06:15:41,695 21269b82 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-11 06:15:41,696 21269b82 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/21269b82/pkb.log
2017-11-11 06:15:41,696 21269b82 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/21269b82/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Jenkins build is back to normal : beam_PerformanceTests_Python #586
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/586/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_Python #585
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/585/display/redirect?page=changes>
Changes:
[robertwb] Add logging and test aborting for timeouts.
[lcwik] Add precommit integration tests to examples/java
[lcwik] Update Gradle PreCommit to publish test results.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 603f86a26122aa611d677e121485ff835c6662d1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 603f86a26122aa611d677e121485ff835c6662d1
Commit message: "Add precommit integration tests to examples/java"
> git rev-list fb41b2950fa2f59f5ff7496af6990c66a715910e # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2008934987287236260.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins964574941314236944.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3760054871174153221.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1422507380351186321.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7184233450391297771.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-22 00:00:14,976 6b318240 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6b318240/pkb.log
2017-11-22 00:00:14,976 6b318240 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-22 00:00:14,976 6b318240 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-22 00:00:15,227 6b318240 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-22 00:00:15,227 6b318240 MainThread INFO Initializing the edw service decoder
2017-11-22 00:00:15,377 6b318240 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-22 00:00:15,380 6b318240 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-22 00:00:15,381 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-22 00:00:22,620 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-22 00:06:26,903 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-22 00:06:26,913 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-22 00:16:26,917 6b318240 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-22 00:16:26,927 6b318240 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-22 00:16:26,928 6b318240 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-22 00:16:26,929 6b318240 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-22 00:16:26,931 6b318240 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-22 00:16:26,945 6b318240 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-22 00:16:26,946 6b318240 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6b318240/pkb.log
2017-11-22 00:16:26,946 6b318240 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6b318240/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #584
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/584/display/redirect>
------------------------------------------
GitHub pull request #4158 of commit c8a5187cd2850613d2881909363e32b19d1155c3, no merge conflicts.
Setting status of c8a5187cd2850613d2881909363e32b19d1155c3 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/584/ and message: 'Build started sha1 is merged.'
Using context: Jenkins: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/4158/*:refs/remotes/origin/pr/4158/*
> git rev-parse refs/remotes/origin/pr/4158/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/4158/merge^{commit} # timeout=10
Checking out Revision c4d3f44854ff8537dea2121622d73aa31bdd8c88 (refs/remotes/origin/pr/4158/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f c4d3f44854ff8537dea2121622d73aa31bdd8c88
Commit message: "Merge c8a5187cd2850613d2881909363e32b19d1155c3 into e9d746a34a82791571b002a78c62e9f46efd4253"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5993992038028145613.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2050962319129494155.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins306321548366558932.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8931499395194623189.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5579537586103628955.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 23:41:09,835 6b83eb7e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6b83eb7e/pkb.log
2017-11-21 23:41:09,836 6b83eb7e MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 23:41:09,837 6b83eb7e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 23:41:10,458 6b83eb7e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 23:41:10,469 6b83eb7e MainThread INFO Initializing the edw service decoder
2017-11-21 23:41:10,967 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 23:41:10,981 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 23:41:10,982 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 23:41:23,047 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 23:48:48,571 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 23:48:48,581 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 23:58:48,586 6b83eb7e Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-21 23:58:48,590 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-21 23:58:48,591 6b83eb7e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 23:58:48,592 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 23:58:48,594 6b83eb7e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 23:58:48,617 6b83eb7e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 23:58:48,617 6b83eb7e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6b83eb7e/pkb.log
2017-11-21 23:58:48,617 6b83eb7e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6b83eb7e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #583
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/583/display/redirect?page=changes>
Changes:
[robertwb] Add option to run PTransform steps from PipelineRunner
[robertwb] Minor cleanup.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision fb41b2950fa2f59f5ff7496af6990c66a715910e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f fb41b2950fa2f59f5ff7496af6990c66a715910e
Commit message: "Closes #4123"
> git rev-list ed40093d8bf47d23e7215d177415f26bfba83fea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7240721229410407952.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2333467120525341955.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2211792954215740008.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7263338426728025283.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1821733305915587363.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 18:27:08,967 e3c22b77 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/e3c22b77/pkb.log
2017-11-21 18:27:08,968 e3c22b77 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 18:27:08,969 e3c22b77 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 18:27:09,344 e3c22b77 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 18:27:09,345 e3c22b77 MainThread INFO Initializing the edw service decoder
2017-11-21 18:27:09,571 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 18:27:09,575 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 18:27:09,576 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 18:27:21,239 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 18:43:35,080 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 18:43:35,090 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/e3c22b77/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 18:43:35,421 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/e3c22b77/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/e3c22b77/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-21 18:43:35,422 e3c22b77 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 18:43:35,437 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 18:43:35,439 e3c22b77 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 18:43:35,458 e3c22b77 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 18:43:35,459 e3c22b77 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/e3c22b77/pkb.log
2017-11-21 18:43:35,459 e3c22b77 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/e3c22b77/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #582
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/582/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed40093d8bf47d23e7215d177415f26bfba83fea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed40093d8bf47d23e7215d177415f26bfba83fea
Commit message: "Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits."
> git rev-list ed40093d8bf47d23e7215d177415f26bfba83fea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5597192447227310593.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2750295580613019511.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4752303249881969147.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1929723363690529870.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7537600052014652448.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 12:00:18,094 0256554e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0256554e/pkb.log
2017-11-21 12:00:18,095 0256554e MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 12:00:18,095 0256554e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 12:00:18,337 0256554e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 12:00:18,338 0256554e MainThread INFO Initializing the edw service decoder
2017-11-21 12:00:18,516 0256554e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 12:00:18,519 0256554e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 12:00:18,519 0256554e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 12:00:24,998 0256554e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 12:06:40,615 0256554e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 12:06:40,627 0256554e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 12:16:40,632 0256554e Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-21 12:16:40,639 0256554e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-21 12:16:40,639 0256554e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 12:16:40,640 0256554e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 12:16:40,642 0256554e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 12:16:40,696 0256554e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 12:16:40,697 0256554e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0256554e/pkb.log
2017-11-21 12:16:40,698 0256554e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0256554e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #581
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/581/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed40093d8bf47d23e7215d177415f26bfba83fea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed40093d8bf47d23e7215d177415f26bfba83fea
Commit message: "Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits."
> git rev-list ed40093d8bf47d23e7215d177415f26bfba83fea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5133738960247356081.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5050038049388389462.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2667335010300677326.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6647271871107147993.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3424496056567103648.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 06:00:26,591 ca162ee3 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ca162ee3/pkb.log
2017-11-21 06:00:26,592 ca162ee3 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 06:00:26,592 ca162ee3 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 06:00:26,884 ca162ee3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 06:00:26,884 ca162ee3 MainThread INFO Initializing the edw service decoder
2017-11-21 06:00:27,076 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 06:00:27,080 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 06:00:27,081 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 06:00:33,554 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 06:07:48,128 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 06:07:48,139 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 06:17:48,144 ca162ee3 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-21 06:17:48,149 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-21 06:17:48,150 ca162ee3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 06:17:48,150 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 06:17:48,152 ca162ee3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 06:17:48,184 ca162ee3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 06:17:48,185 ca162ee3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ca162ee3/pkb.log
2017-11-21 06:17:48,185 ca162ee3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ca162ee3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #580
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/580/display/redirect?page=changes>
Changes:
[lcwik] Enable Java Gradle Precommit in Jenkins
[lcwik] Simplify maintainability by generating tasks and command line from list
[lcwik] Limit the commit status length to 255 characters to honor Github limit
[lcwik] Revert PR/3976, the Jenkins Gradle plugin expects the Gradle Wrapper to
[lcwik] Sync with master and add new subproject for beam-runners-local-java-core
[kirpichov] Improved validation of DynamicDestinations results in BigQueryIO.write
[chamikara] Dataflow and PerfKit profiles; hash for 100.000.000 lines
[lcwik] Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other
[lcwik] Update the location of the root build directory to coincide with the
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed40093d8bf47d23e7215d177415f26bfba83fea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed40093d8bf47d23e7215d177415f26bfba83fea
Commit message: "Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits."
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5230896743662012013.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8667876498156484861.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1337895185962056481.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8231665045133042800.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
sdks/python/[gcp,test] should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #579
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/579/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5373597544739417499.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5592195336555117248.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3763933392533803781.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4999056484096096630.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1784607244369668976.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-20 12:00:23,001 0ac4309d MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0ac4309d/pkb.log
2017-11-20 12:00:23,001 0ac4309d MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-20 12:00:23,002 0ac4309d MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-20 12:00:23,200 0ac4309d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-20 12:00:23,201 0ac4309d MainThread INFO Initializing the edw service decoder
2017-11-20 12:00:23,356 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-20 12:00:23,359 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-20 12:00:23,359 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-20 12:00:30,653 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-20 12:06:35,077 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-20 12:06:35,088 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-20 12:16:35,095 0ac4309d Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-20 12:16:35,099 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-20 12:16:35,100 0ac4309d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-20 12:16:35,102 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-20 12:16:35,104 0ac4309d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-20 12:16:35,154 0ac4309d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-20 12:16:35,155 0ac4309d MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0ac4309d/pkb.log
2017-11-20 12:16:35,155 0ac4309d MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0ac4309d/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #578
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/578/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins767942611611723038.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4174273406739557118.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins761864003373348100.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5839146230927018378.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1249975475942734405.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-20 06:00:16,653 f5729124 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f5729124/pkb.log
2017-11-20 06:00:16,653 f5729124 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-20 06:00:16,653 f5729124 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-20 06:00:16,781 f5729124 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-20 06:00:16,782 f5729124 MainThread INFO Initializing the edw service decoder
2017-11-20 06:00:16,951 f5729124 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-20 06:00:16,954 f5729124 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-20 06:00:16,954 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-20 06:00:23,564 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-20 06:06:41,707 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-20 06:06:41,718 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-20 06:16:41,723 f5729124 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-20 06:16:41,730 f5729124 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-20 06:16:41,730 f5729124 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-20 06:16:41,731 f5729124 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-20 06:16:41,733 f5729124 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-20 06:16:41,759 f5729124 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-20 06:16:41,760 f5729124 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f5729124/pkb.log
2017-11-20 06:16:41,760 f5729124 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f5729124/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #577
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/577/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5014233780848433318.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3041297463291230105.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2773657514736843752.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1538629637840030819.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7506920532129854367.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-20 00:00:25,442 69fe8637 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/69fe8637/pkb.log
2017-11-20 00:00:25,442 69fe8637 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-20 00:00:25,443 69fe8637 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-20 00:00:25,828 69fe8637 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-20 00:00:25,828 69fe8637 MainThread INFO Initializing the edw service decoder
2017-11-20 00:00:26,016 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-20 00:00:26,019 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-20 00:00:26,020 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-20 00:00:31,550 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-20 00:07:44,579 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-20 00:07:44,589 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-20 00:17:44,594 69fe8637 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-20 00:17:44,600 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-20 00:17:44,600 69fe8637 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-20 00:17:44,601 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-20 00:17:44,603 69fe8637 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-20 00:17:44,636 69fe8637 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-20 00:17:44,637 69fe8637 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/69fe8637/pkb.log
2017-11-20 00:17:44,637 69fe8637 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/69fe8637/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #576
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/576/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5520766847453943497.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6226155290360226877.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4145240403247521989.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7971707459832702207.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7480621607496451934.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 18:00:17,317 85b96e9a MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/85b96e9a/pkb.log
2017-11-19 18:00:17,317 85b96e9a MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 18:00:17,318 85b96e9a MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 18:00:17,559 85b96e9a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 18:00:17,560 85b96e9a MainThread INFO Initializing the edw service decoder
2017-11-19 18:00:17,712 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 18:00:17,715 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 18:00:17,715 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 18:00:24,552 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 18:06:30,931 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 18:06:30,942 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 18:16:30,947 85b96e9a Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-19 18:16:30,954 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-19 18:16:30,955 85b96e9a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 18:16:30,956 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 18:16:30,958 85b96e9a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 18:16:30,985 85b96e9a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 18:16:30,985 85b96e9a MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/85b96e9a/pkb.log
2017-11-19 18:16:30,986 85b96e9a MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/85b96e9a/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #575
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/575/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1471960231046679688.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7811412306516364141.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9167470683863290579.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3869762181712991063.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins44924488039484535.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 12:00:16,109 59025a31 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/59025a31/pkb.log
2017-11-19 12:00:16,109 59025a31 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 12:00:16,110 59025a31 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 12:00:16,324 59025a31 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 12:00:16,324 59025a31 MainThread INFO Initializing the edw service decoder
2017-11-19 12:00:16,497 59025a31 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 12:00:16,501 59025a31 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 12:00:16,502 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 12:00:23,673 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 12:06:29,682 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 12:06:29,693 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 12:16:29,698 59025a31 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-19 12:16:29,704 59025a31 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-19 12:16:29,705 59025a31 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 12:16:29,706 59025a31 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 12:16:29,707 59025a31 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 12:16:29,759 59025a31 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 12:16:29,760 59025a31 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/59025a31/pkb.log
2017-11-19 12:16:29,760 59025a31 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/59025a31/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #574
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/574/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins32354818368900436.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2823221348356969534.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5300710995425967786.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4735169387755430985.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6203202532533460797.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 06:00:27,531 bfd449c6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/bfd449c6/pkb.log
2017-11-19 06:00:27,532 bfd449c6 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 06:00:27,533 bfd449c6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 06:00:28,241 bfd449c6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 06:00:28,241 bfd449c6 MainThread INFO Initializing the edw service decoder
2017-11-19 06:00:28,729 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 06:00:28,733 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 06:00:28,734 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 06:00:46,411 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 06:17:49,609 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 06:17:49,716 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/bfd449c6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 06:17:50,311 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/bfd449c6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/bfd449c6/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-19 06:17:50,312 bfd449c6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 06:17:50,313 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 06:17:50,315 bfd449c6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 06:17:50,341 bfd449c6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 06:17:50,342 bfd449c6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/bfd449c6/pkb.log
2017-11-19 06:17:50,342 bfd449c6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/bfd449c6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #573
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/573/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6432721446071832801.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1749592125091995284.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3868056377492935426.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins278161475843902295.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.3.0.dev)
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Collecting futures<4.0.0,>=3.1.1 (from apache-beam==2.3.0.dev)
Using cached futures-3.1.1-py2-none-any.whl
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: pyvcf, futures, apache-beam
Found existing installation: futures 3.0.5
Uninstalling futures-3.0.5:
Successfully uninstalled futures-3.0.5
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam futures-3.1.1 pyvcf-0.6.8
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2716840393397482088.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 00:01:00,693 a435512b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/a435512b/pkb.log
2017-11-19 00:01:00,693 a435512b MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 00:01:00,694 a435512b MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 00:01:01,137 a435512b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 00:01:01,137 a435512b MainThread INFO Initializing the edw service decoder
2017-11-19 00:01:01,329 a435512b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 00:01:01,332 a435512b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 00:01:01,333 a435512b MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 00:01:07,635 a435512b MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 00:08:20,425 a435512b MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 00:08:20,437 a435512b MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 00:18:20,442 a435512b Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-19 00:18:20,446 a435512b MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-19 00:18:20,447 a435512b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 00:18:20,448 a435512b MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 00:18:20,450 a435512b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 00:18:20,473 a435512b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 00:18:20,473 a435512b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/a435512b/pkb.log
2017-11-19 00:18:20,474 a435512b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/a435512b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #572
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/572/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6067668881080642432.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8421980976207251306.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6665273338505486915.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1590361676075629645.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 521, in _prepare_file
req_to_install.check_if_exists()
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 1036, in check_if_exists
self.req.name
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 558, in get_distribution
dist = get_provider(dist)
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 432, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 968, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 859, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
ContextualVersionConflict: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures<4.0.0,>=3.1.1'), set(['apache-beam']))
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #571
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/571/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9111897738739701955.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2255391586014356900.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4858012336430098801.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9116202591903373260.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4006141163560924042.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-18 12:00:16,779 18bee801 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/18bee801/pkb.log
2017-11-18 12:00:16,779 18bee801 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-18 12:00:16,780 18bee801 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-18 12:00:16,982 18bee801 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-18 12:00:16,982 18bee801 MainThread INFO Initializing the edw service decoder
2017-11-18 12:00:17,135 18bee801 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-18 12:00:17,138 18bee801 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-18 12:00:17,138 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-18 12:00:24,065 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-18 12:06:43,199 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-18 12:06:43,209 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/18bee801/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-18 12:06:43,352 18bee801 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/18bee801/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/18bee801/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-18 12:06:43,353 18bee801 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-18 12:06:43,354 18bee801 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-18 12:06:43,355 18bee801 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-18 12:06:43,393 18bee801 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-18 12:06:43,394 18bee801 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/18bee801/pkb.log
2017-11-18 12:06:43,394 18bee801 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/18bee801/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #570
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/570/display/redirect?page=changes>
Changes:
[kedin] [Nexmark] Extract GeneratorCheckpoint into a separate class. Move
[kedin] [Nexmark] Extract PersonGenerator, StringsGenerator, LongGenerator from
[kedin] [Nexmark] Extract AuctionGenerator, PriceGenerator from Generator
[kedin] [Nexmark] Extract BidGenerator from Generator
[lcwik] Move AVRO files to avro source location
[lcwik] Initial gradle build definitions for all java based modules which is
[lcwik] Add license headers to all build.gradle files.
[lcwik] Add provided scope plugin and migrate compileOnly/testCompileOnly to
[lcwik] Add the ability to list dependencies recursively. Also ban the
[lcwik] Update comments
[lcwik] Configure the checkstyle plugin
[lcwik] Add support to run the Apache Rat plugin Add gradle generated files to
[lcwik] Adding Python gradle changes
[lcwik] Fix python tasks to not re-use the default gradle lifecycle names.
[lcwik] Add test task dependencies for testCython and testGcp and also add lint
[lcwik] Create build_rules file moving out specific plugins to them.
[lcwik] Add missing build_rules file.
[lcwik] Ignore test categories NeedsRunner, ValidatesRunner, and all *IT tests
[lcwik] Add code style validation and auto-formatting to Gradle build.
[lcwik] Add beam-sdks-java-core dependency to beam-local-artifact-service
[lcwik] Add gradle build for sdks/go
[lcwik] Get evaluation dependency order fixed because of dependency on
[lcwik] Update comments on root project
[lcwik] Set beamUseDummyRunner = true
[lcwik] Move spotless plugin to delegate config
[lcwik] Fix spotless config
[lcwik] Change hamcrest and junit to testCompile in
[lcwik] Add findbugs to the build
[lcwik] Add test to check phase
[lcwik] Filter resources to set dataflow environrment versions in
[lcwik] Force dependency resolution of common Java dependencies
[lcwik] Filter sdk.properties, set pom.version and build timestamp
[lcwik] Use dummy runner in tests in google-cloud-dataflow-java
[lcwik] Add test dependecy on beam-sdks-java-core to
[lcwik] Migrate findbugs to be configurable from the JavaNatureConfiguration
[lcwik] Setup several maven-surefire-plugin configurations
[lcwik] Remove added sdk-core dependencies because of
[lcwik] Fix provided scope in sdks/java/extensions/sorter
[lcwik] Ban hamcrest-all/mockito-all and migrate to hamcrest-core/mockito-core
[lcwik] Add gradle rule for python container
[lcwik] Fix python container gradle build
[lcwik] Fix hadoop-file-system test dependencies Explain why hcatalog is failing
[lcwik] Add license header to sdks/python/build.gradle
[lcwik] Get spark runner unit tests to pass.
[lcwik] Define the correct versions of apex-core Add comments to hcatalog/hbase
[lcwik] Simplify ApexYarnLauncher matching Attempt to create dependency tree
[lcwik] Enable configuraton on demand by default Get ApexYarnLauncher test to
[lcwik] Fix sdks/java/core testCompile dependency to bring in
[lcwik] Re-enable Python SDK
[lcwik] Disable findbugs within sdks/java/io/kafka to match findbugs
[lcwik] Add a task dependency visualization plugin. Resolve a spurious error
[lcwik] Fix classpath bundling issue caused by Hive and its dependencies
[lcwik] Define Tika dependencies version as variable
[lcwik] Rebase on apache/master and add new subprojects, update versions of
[lcwik] Skip test failures in hadoop related modules due to dependency
[lcwik] Add Jacoco plugin
[lcwik] Add Gradle builds for gcemd and gcsproxy
[lcwik] Add Java SDK Container Gradle build file
[lcwik] Create uber jar of the Java SDK harness and copy it into the location
[lcwik] Make docker tasks correctly depend on required build tasks
[lcwik] Fix runners/gcp/* go package paths
[lcwik] Add shadow configuration for sdks/java/core updating all other projects
[lcwik] Configure default shading profile on all java sub-projects to shade away
[lcwik] Shade direct-java and sql projects. Update repackaged locations to
[lcwik] Fix gcemd and gcsproxy builds. Fix whitespace formatting.
[lcwik] Enforce that gogradle dependency resolution/installation is done
[lcwik] Also ensure that resolving test dependencies are done sequentially with
[lcwik] Use valid repackage path on shadow relocations Fix Spark runners tests
[lcwik] Move user.avsc to directory which corresponds with namespace under
[lcwik] Add comments to various parts of build_rules.gradle
[lcwik] Fix excludes to correctly match the .idea/ directory.
[lcwik] Update build.gradle files to correspond with pom.xml changes within
[lcwik] Enable FFMP -> JavaCC path to generate Beam SQL parser for
[tgroh] Add a Local Java Core Module
[tgroh] Move StructuralKey to the local execution module
[tgroh] Add A TransformExecutorFactory
[chamikara] Clarify documentation on support for reading from/writing to time
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list f10399d7c374d0342fdac524c8a2ddf4d7ec5cd3 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2133352925076915801.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7168219796139475913.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins988525984695506756.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins544213913297504256.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7832708401297371404.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-18 06:00:32,127 73ec6fc3 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/73ec6fc3/pkb.log
2017-11-18 06:00:32,127 73ec6fc3 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-18 06:00:32,128 73ec6fc3 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-18 06:00:32,379 73ec6fc3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-18 06:00:32,379 73ec6fc3 MainThread INFO Initializing the edw service decoder
2017-11-18 06:00:32,561 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-18 06:00:32,564 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-18 06:00:32,565 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-18 06:00:39,075 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-18 06:08:21,383 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-18 06:08:21,397 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/73ec6fc3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-18 06:08:21,700 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/73ec6fc3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/73ec6fc3/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-18 06:08:21,701 73ec6fc3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-18 06:08:21,702 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-18 06:08:21,703 73ec6fc3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-18 06:08:21,749 73ec6fc3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-18 06:08:21,750 73ec6fc3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/73ec6fc3/pkb.log
2017-11-18 06:08:21,750 73ec6fc3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/73ec6fc3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #569
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/569/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3206] Shut down executor when spark runner finishes
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f10399d7c374d0342fdac524c8a2ddf4d7ec5cd3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f10399d7c374d0342fdac524c8a2ddf4d7ec5cd3
Commit message: "This closes #4138"
> git rev-list 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins343152140608123410.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins88728062273622039.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6844791055540984298.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3694502064599885753.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7140621039230471732.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 18:00:26,432 714bba78 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/714bba78/pkb.log
2017-11-17 18:00:26,433 714bba78 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 18:00:26,434 714bba78 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 18:00:27,042 714bba78 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 18:00:27,043 714bba78 MainThread INFO Initializing the edw service decoder
2017-11-17 18:00:27,266 714bba78 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 18:00:27,270 714bba78 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 18:00:27,271 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 18:00:40,271 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 18:18:00,529 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 18:18:00,552 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/714bba78/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 18:18:00,872 714bba78 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/714bba78/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/714bba78/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 18:18:00,873 714bba78 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 18:18:00,874 714bba78 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 18:18:00,877 714bba78 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 18:18:00,891 714bba78 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 18:18:00,891 714bba78 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/714bba78/pkb.log
2017-11-17 18:18:00,891 714bba78 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/714bba78/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #568
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/568/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee
Commit message: "This closes #4057: [BEAM-3035] Extract ReifyTimestampsAndWindows"
> git rev-list 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1769809359256019078.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3303174299824323028.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1771786221137633440.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins642950221683200409.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7981304723880363437.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 12:00:29,475 5fdb6c75 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/5fdb6c75/pkb.log
2017-11-17 12:00:29,476 5fdb6c75 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 12:00:29,477 5fdb6c75 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 12:00:29,956 5fdb6c75 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 12:00:29,956 5fdb6c75 MainThread INFO Initializing the edw service decoder
2017-11-17 12:00:30,326 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 12:00:30,330 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 12:00:30,331 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 12:00:42,818 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 12:13:14,706 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 12:13:14,720 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5fdb6c75/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 12:13:14,974 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5fdb6c75/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/5fdb6c75/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 12:13:14,975 5fdb6c75 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 12:13:14,976 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 12:13:14,978 5fdb6c75 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 12:13:15,008 5fdb6c75 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 12:13:15,008 5fdb6c75 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/5fdb6c75/pkb.log
2017-11-17 12:13:15,008 5fdb6c75 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/5fdb6c75/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #567
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/567/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee
Commit message: "This closes #4057: [BEAM-3035] Extract ReifyTimestampsAndWindows"
> git rev-list 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5322896415489438698.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8955443008791098663.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5560883738286550035.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4209878166945532977.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4439886988845878833.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 06:00:19,288 0fc1b167 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0fc1b167/pkb.log
2017-11-17 06:00:19,288 0fc1b167 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 06:00:19,289 0fc1b167 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 06:00:19,748 0fc1b167 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 06:00:19,749 0fc1b167 MainThread INFO Initializing the edw service decoder
2017-11-17 06:00:19,974 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 06:00:19,978 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 06:00:19,979 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 06:00:27,897 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 06:11:11,083 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 06:11:11,118 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0fc1b167/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 06:11:11,466 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0fc1b167/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/0fc1b167/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 06:11:11,466 0fc1b167 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 06:11:11,515 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 06:11:11,518 0fc1b167 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 06:11:11,555 0fc1b167 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 06:11:11,556 0fc1b167 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0fc1b167/pkb.log
2017-11-17 06:11:11,556 0fc1b167 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0fc1b167/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #566
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/566/display/redirect?page=changes>
Changes:
[valentyn] Fix user_score example.
[lcwik] [BEAM-3202] Ensure that PipelineOptions.getOptionsId is always
[kirpichov] [BEAM-3035] Introduces Reify transform
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee
Commit message: "This closes #4057: [BEAM-3035] Extract ReifyTimestampsAndWindows"
> git rev-list 92013ec9e659228dd2ea66a28e2d83e7f4795b61 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9021074132772576932.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1454103446174442545.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5973398805072968925.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4032666837727789652.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7329995538676718832.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 00:00:37,168 b3bd6d5f MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b3bd6d5f/pkb.log
2017-11-17 00:00:37,169 b3bd6d5f MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 00:00:37,170 b3bd6d5f MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 00:00:37,568 b3bd6d5f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 00:00:37,569 b3bd6d5f MainThread INFO Initializing the edw service decoder
2017-11-17 00:00:37,884 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 00:00:37,889 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 00:00:37,890 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 00:00:50,820 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 00:13:50,350 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 00:13:50,366 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b3bd6d5f/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 00:13:50,784 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b3bd6d5f/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/b3bd6d5f/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 00:13:50,784 b3bd6d5f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 00:13:50,785 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 00:13:50,788 b3bd6d5f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 00:13:50,798 b3bd6d5f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 00:13:50,798 b3bd6d5f MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b3bd6d5f/pkb.log
2017-11-17 00:13:50,798 b3bd6d5f MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b3bd6d5f/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #565
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/565/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92013ec9e659228dd2ea66a28e2d83e7f4795b61 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92013ec9e659228dd2ea66a28e2d83e7f4795b61
Commit message: "This closes #4137: [BEAM-3195] Specifies numShards on windowed writes examples, as it is now required"
> git rev-list 92013ec9e659228dd2ea66a28e2d83e7f4795b61 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3121235113020595809.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1648897548745660183.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4103012481848862924.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2433649201481481848.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8493788332974716657.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 18:00:25,410 8954b74b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/8954b74b/pkb.log
2017-11-16 18:00:25,410 8954b74b MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 18:00:25,411 8954b74b MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 18:00:25,863 8954b74b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 18:00:25,864 8954b74b MainThread INFO Initializing the edw service decoder
2017-11-16 18:00:26,050 8954b74b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 18:00:26,053 8954b74b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 18:00:26,054 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 18:00:34,157 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 18:11:05,310 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 18:11:05,326 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8954b74b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 18:11:05,640 8954b74b MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8954b74b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/8954b74b/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 18:11:05,640 8954b74b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 18:11:05,641 8954b74b MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 18:11:05,644 8954b74b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 18:11:05,695 8954b74b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 18:11:05,696 8954b74b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/8954b74b/pkb.log
2017-11-16 18:11:05,696 8954b74b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/8954b74b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #564
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/564/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92013ec9e659228dd2ea66a28e2d83e7f4795b61 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92013ec9e659228dd2ea66a28e2d83e7f4795b61
Commit message: "This closes #4137: [BEAM-3195] Specifies numShards on windowed writes examples, as it is now required"
> git rev-list 92013ec9e659228dd2ea66a28e2d83e7f4795b61 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins942289546882004076.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3611155006219490356.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3822713513701605842.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1001870562125754694.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4850996491855444649.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 12:00:22,634 81071ae6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/81071ae6/pkb.log
2017-11-16 12:00:22,635 81071ae6 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 12:00:22,636 81071ae6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 12:00:23,059 81071ae6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 12:00:23,059 81071ae6 MainThread INFO Initializing the edw service decoder
2017-11-16 12:00:23,325 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 12:00:23,329 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 12:00:23,330 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 12:00:36,147 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 12:12:37,351 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 12:12:37,392 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/81071ae6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 12:12:37,712 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/81071ae6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/81071ae6/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 12:12:37,712 81071ae6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 12:12:37,713 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 12:12:37,716 81071ae6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 12:12:37,733 81071ae6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 12:12:37,733 81071ae6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/81071ae6/pkb.log
2017-11-16 12:12:37,733 81071ae6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/81071ae6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #563
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/563/display/redirect?page=changes>
Changes:
[kirpichov] Specifies numShards on windowed writes examples, as it is now required
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92013ec9e659228dd2ea66a28e2d83e7f4795b61 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92013ec9e659228dd2ea66a28e2d83e7f4795b61
Commit message: "This closes #4137: [BEAM-3195] Specifies numShards on windowed writes examples, as it is now required"
> git rev-list c3a96bf3d9d40f7636d8ac62326aa75e93c0b11c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6193826000221498997.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6867784704250554535.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1664628939435077919.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6317801227600658056.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5743809504303722366.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 06:00:37,736 f1c8efd8 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f1c8efd8/pkb.log
2017-11-16 06:00:37,737 f1c8efd8 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 06:00:37,738 f1c8efd8 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 06:00:38,317 f1c8efd8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 06:00:38,318 f1c8efd8 MainThread INFO Initializing the edw service decoder
2017-11-16 06:00:38,626 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 06:00:38,631 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 06:00:38,631 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 06:00:52,509 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 06:14:45,636 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 06:14:45,661 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f1c8efd8/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 06:14:46,347 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f1c8efd8/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/f1c8efd8/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 06:14:46,348 f1c8efd8 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 06:14:46,358 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 06:14:46,361 f1c8efd8 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 06:14:46,385 f1c8efd8 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 06:14:46,386 f1c8efd8 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f1c8efd8/pkb.log
2017-11-16 06:14:46,386 f1c8efd8 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f1c8efd8/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #562
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/562/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-2926] Migrate to using a trivial multimap materialization within
[lcwik] Rebase and update code to honor findbugs @Nullable/@Nonnull conversion
[lcwik] Update Dataflow worker image to be compatible with side input changes.
[lcwik] Replace the View.As transforms for Dataflow batch because the entire
[lcwik] Fix BatchViewOverrides ViewAsSingleton to apply the combine fn that was
[altay] Fixed adding timestamp and id attributes to pubsub messages
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision c3a96bf3d9d40f7636d8ac62326aa75e93c0b11c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f c3a96bf3d9d40f7636d8ac62326aa75e93c0b11c
Commit message: "This closes #3581"
> git rev-list 7ce0a82b06c29247713c009d32b53fe53fd7ec6d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8976025187010219755.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4509251861334500213.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6514203397664823258.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4091882124589489934.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8378667589799528935.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 00:00:35,522 3ecb3927 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/3ecb3927/pkb.log
2017-11-16 00:00:35,523 3ecb3927 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 00:00:35,524 3ecb3927 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 00:00:36,231 3ecb3927 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 00:00:36,231 3ecb3927 MainThread INFO Initializing the edw service decoder
2017-11-16 00:00:36,619 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 00:00:36,624 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 00:00:36,625 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 00:00:50,512 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 00:13:49,340 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 00:13:49,360 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3ecb3927/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 00:13:49,704 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3ecb3927/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/3ecb3927/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 00:13:49,704 3ecb3927 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 00:13:49,705 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 00:13:49,708 3ecb3927 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 00:13:49,756 3ecb3927 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 00:13:49,756 3ecb3927 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/3ecb3927/pkb.log
2017-11-16 00:13:49,757 3ecb3927 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/3ecb3927/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #561
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/561/display/redirect?page=changes>
Changes:
[chamikara] Updated jdbc's pom.xml
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7ce0a82b06c29247713c009d32b53fe53fd7ec6d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7ce0a82b06c29247713c009d32b53fe53fd7ec6d
Commit message: "This closes #4121"
> git rev-list 209bb6d748bb776cdb5d93729144ba34dd9cb967 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7582666607470916224.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6778610571247056304.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5383684519806896547.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7107198820040444271.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6243242900567046194.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-15 12:00:22,491 fc892df9 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/fc892df9/pkb.log
2017-11-15 12:00:22,491 fc892df9 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-15 12:00:22,492 fc892df9 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-15 12:00:22,917 fc892df9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-15 12:00:22,917 fc892df9 MainThread INFO Initializing the edw service decoder
2017-11-15 12:00:23,227 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-15 12:00:23,232 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-15 12:00:23,233 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-15 12:00:36,717 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-15 12:12:56,301 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-15 12:12:56,320 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/fc892df9/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-15 12:12:56,714 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/fc892df9/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/fc892df9/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-15 12:12:56,714 fc892df9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-15 12:12:56,715 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-15 12:12:56,718 fc892df9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-15 12:12:56,766 fc892df9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-15 12:12:56,767 fc892df9 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/fc892df9/pkb.log
2017-11-15 12:12:56,767 fc892df9 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/fc892df9/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #560
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/560/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-3150] Fix threading for progress reporting in SDKHarness
[robertwb] Update minimum cython version to 0.26.1
[kirpichov] WriteFiles improvements:
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 209bb6d748bb776cdb5d93729144ba34dd9cb967 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 209bb6d748bb776cdb5d93729144ba34dd9cb967
Commit message: "This closes #4124: [BEAM-3169] Fixes a data loss bug in WriteFiles when used with fire-once triggers"
> git rev-list a5254e730065d8b15899b094ee54011797fffaa1 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4917994311947654522.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins255977492571787228.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4206405982811881579.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8568715148996427018.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2198126312727459640.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-15 06:00:36,870 4ca90d93 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/4ca90d93/pkb.log
2017-11-15 06:00:36,871 4ca90d93 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-15 06:00:36,872 4ca90d93 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-15 06:00:37,247 4ca90d93 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-15 06:00:37,247 4ca90d93 MainThread INFO Initializing the edw service decoder
2017-11-15 06:00:37,642 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-15 06:00:37,646 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-15 06:00:37,658 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-15 06:00:50,028 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-15 06:13:53,412 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-15 06:13:53,433 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/4ca90d93/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-15 06:13:53,830 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/4ca90d93/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/4ca90d93/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-15 06:13:53,831 4ca90d93 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-15 06:13:53,832 4ca90d93 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-15 06:13:53,834 4ca90d93 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-15 06:13:53,863 4ca90d93 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-15 06:13:53,863 4ca90d93 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/4ca90d93/pkb.log
2017-11-15 06:13:53,863 4ca90d93 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/4ca90d93/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #559
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/559/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a5254e730065d8b15899b094ee54011797fffaa1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a5254e730065d8b15899b094ee54011797fffaa1
Commit message: "This closes #3988: [BEAM-3052] ReduceFnRunner: Do not manage EOW hold or timer, set GC hold and timer always"
> git rev-list a5254e730065d8b15899b094ee54011797fffaa1 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5280454676584367258.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9168030622354098104.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4457302580537247041.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5286398457891674171.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7836209971775739017.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-15 00:00:17,815 6738619a MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6738619a/pkb.log
2017-11-15 00:00:17,816 6738619a MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-15 00:00:17,816 6738619a MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-15 00:00:18,540 6738619a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-15 00:00:18,541 6738619a MainThread INFO Initializing the edw service decoder
2017-11-15 00:00:18,801 6738619a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-15 00:00:18,819 6738619a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-15 00:00:18,819 6738619a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-15 00:00:36,436 6738619a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-15 00:15:01,311 6738619a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-15 00:15:01,375 6738619a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6738619a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-15 00:15:01,614 6738619a MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6738619a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/6738619a/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-15 00:15:01,614 6738619a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-15 00:15:01,616 6738619a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-15 00:15:01,618 6738619a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-15 00:15:01,659 6738619a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-15 00:15:01,660 6738619a MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6738619a/pkb.log
2017-11-15 00:15:01,660 6738619a MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6738619a/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #558
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/558/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a5254e730065d8b15899b094ee54011797fffaa1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a5254e730065d8b15899b094ee54011797fffaa1
Commit message: "This closes #3988: [BEAM-3052] ReduceFnRunner: Do not manage EOW hold or timer, set GC hold and timer always"
> git rev-list a5254e730065d8b15899b094ee54011797fffaa1 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8604132266081766841.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5402779791475217465.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7337748322331290040.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9060806786643880005.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6270298113941179408.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-14 18:00:19,669 90a67e08 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/90a67e08/pkb.log
2017-11-14 18:00:19,674 90a67e08 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-14 18:00:19,675 90a67e08 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-14 18:00:20,191 90a67e08 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-14 18:00:20,192 90a67e08 MainThread INFO Initializing the edw service decoder
2017-11-14 18:00:20,452 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-14 18:00:20,456 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-14 18:00:20,457 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-14 18:00:29,934 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-14 18:14:25,267 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-14 18:14:25,309 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/90a67e08/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-14 18:14:25,501 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/90a67e08/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/90a67e08/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-14 18:14:25,502 90a67e08 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-14 18:14:25,503 90a67e08 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-14 18:14:25,505 90a67e08 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-14 18:14:25,515 90a67e08 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-14 18:14:25,515 90a67e08 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/90a67e08/pkb.log
2017-11-14 18:14:25,515 90a67e08 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/90a67e08/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #557
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/557/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a5254e730065d8b15899b094ee54011797fffaa1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a5254e730065d8b15899b094ee54011797fffaa1
Commit message: "This closes #3988: [BEAM-3052] ReduceFnRunner: Do not manage EOW hold or timer, set GC hold and timer always"
> git rev-list a5254e730065d8b15899b094ee54011797fffaa1 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5680871708644826985.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4991299418139157733.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2946856032098368137.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6401676500596866806.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8082933889760822472.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-14 06:00:28,415 4773272f MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/4773272f/pkb.log
2017-11-14 06:00:28,415 4773272f MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-14 06:00:28,416 4773272f MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-14 06:00:29,443 4773272f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-14 06:00:29,450 4773272f MainThread INFO Initializing the edw service decoder
2017-11-14 06:00:29,838 4773272f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-14 06:00:29,852 4773272f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-14 06:00:29,853 4773272f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-14 06:00:44,487 4773272f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-14 06:15:10,049 4773272f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-14 06:15:10,074 4773272f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/4773272f/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-14 06:15:10,297 4773272f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/4773272f/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/4773272f/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-14 06:15:10,297 4773272f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-14 06:15:10,299 4773272f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-14 06:15:10,301 4773272f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-14 06:15:10,349 4773272f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-14 06:15:10,350 4773272f MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/4773272f/pkb.log
2017-11-14 06:15:10,350 4773272f MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/4773272f/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #556
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/556/display/redirect?page=changes>
Changes:
[kenn] Stage the portable pipeline; put URL in pipeline options
[chamikara] Integrate bundle retry code for the DirectRunner
[ccy] Fix flakiness in sideinputs_test
[kenn] Fix expectations in CombineTest
[kenn] Tidy a troublesome TestStreamTest
[kenn] Truncate the very last fixed window if it goes beyond representable time
[kenn] ReduceFnRunner: Do not manage EOW hold or timer, set GC hold and timer
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a5254e730065d8b15899b094ee54011797fffaa1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a5254e730065d8b15899b094ee54011797fffaa1
Commit message: "This closes #3988: [BEAM-3052] ReduceFnRunner: Do not manage EOW hold or timer, set GC hold and timer always"
> git rev-list 2b4a6b5d5cb455fe73971fde4caa7d1446cd55aa # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8769179241073125942.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6421822412316727374.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7959916717141665869.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4720338680646903812.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins183820856156581343.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-14 00:00:41,686 ce786791 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ce786791/pkb.log
2017-11-14 00:00:41,687 ce786791 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-14 00:00:41,688 ce786791 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-14 00:00:42,024 ce786791 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-14 00:00:42,025 ce786791 MainThread INFO Initializing the edw service decoder
2017-11-14 00:00:42,305 ce786791 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-14 00:00:42,309 ce786791 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-14 00:00:42,310 ce786791 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-14 00:00:54,551 ce786791 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-14 00:15:18,428 ce786791 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-14 00:15:18,506 ce786791 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ce786791/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-14 00:15:20,111 ce786791 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ce786791/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/ce786791/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-14 00:15:20,112 ce786791 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-14 00:15:20,143 ce786791 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-14 00:15:20,145 ce786791 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-14 00:15:20,176 ce786791 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-14 00:15:20,176 ce786791 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ce786791/pkb.log
2017-11-14 00:15:20,176 ce786791 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ce786791/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #555
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/555/display/redirect?page=changes>
Changes:
[xumingmingv] support topicPartition in BeamKafkaTable
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 2b4a6b5d5cb455fe73971fde4caa7d1446cd55aa (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 2b4a6b5d5cb455fe73971fde4caa7d1446cd55aa
Commit message: "This closes #4115"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins813920738944535188.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4522735723633700155.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins757082198714946499.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1168611204358401089.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2428105172647828063.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-13 18:15:01,206 cf57b363 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/cf57b363/pkb.log
2017-11-13 18:15:01,206 cf57b363 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-13 18:15:01,207 cf57b363 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-13 18:15:01,610 cf57b363 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-13 18:15:01,610 cf57b363 MainThread INFO Initializing the edw service decoder
2017-11-13 18:15:01,771 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-13 18:15:01,774 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-13 18:15:01,774 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-13 18:15:09,031 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-13 18:23:40,897 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-13 18:23:40,908 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/cf57b363/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-13 18:23:41,306 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/cf57b363/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/cf57b363/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-13 18:23:41,306 cf57b363 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-13 18:23:41,308 cf57b363 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-13 18:23:41,310 cf57b363 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-13 18:23:41,355 cf57b363 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-13 18:23:41,355 cf57b363 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/cf57b363/pkb.log
2017-11-13 18:23:41,356 cf57b363 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/cf57b363/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #554
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/554/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5216829711633960062.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins799767504342319348.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4522936834530418042.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8182085694676591845.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6278161561372707143.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-13 06:00:37,647 1a599cfa MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/1a599cfa/pkb.log
2017-11-13 06:00:37,656 1a599cfa MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-13 06:00:37,657 1a599cfa MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-13 06:00:38,069 1a599cfa MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-13 06:00:38,070 1a599cfa MainThread INFO Initializing the edw service decoder
2017-11-13 06:00:38,269 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-13 06:00:38,276 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-13 06:00:38,277 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-13 06:00:47,827 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-13 06:22:09,524 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-13 06:22:09,539 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1a599cfa/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-13 06:22:09,810 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1a599cfa/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/1a599cfa/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-13 06:22:09,811 1a599cfa MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-13 06:22:09,811 1a599cfa MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-13 06:22:09,813 1a599cfa MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-13 06:22:09,854 1a599cfa MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-13 06:22:09,855 1a599cfa MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/1a599cfa/pkb.log
2017-11-13 06:22:09,855 1a599cfa MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/1a599cfa/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #553
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/553/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3492318021979492637.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3671509680136512908.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4997885982231297469.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1001187242812439099.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8222370205773096743.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-13 00:00:14,891 d4488205 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/d4488205/pkb.log
2017-11-13 00:00:14,895 d4488205 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-13 00:00:14,895 d4488205 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-13 00:00:15,149 d4488205 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-13 00:00:15,150 d4488205 MainThread INFO Initializing the edw service decoder
2017-11-13 00:00:15,305 d4488205 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-13 00:00:15,308 d4488205 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-13 00:00:15,308 d4488205 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-13 00:00:21,416 d4488205 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-13 00:15:13,064 d4488205 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-13 00:15:13,075 d4488205 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/d4488205/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-13 00:15:13,225 d4488205 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/d4488205/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/d4488205/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-13 00:15:13,226 d4488205 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-13 00:15:13,226 d4488205 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-13 00:15:13,228 d4488205 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-13 00:15:13,282 d4488205 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-13 00:15:13,283 d4488205 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/d4488205/pkb.log
2017-11-13 00:15:13,283 d4488205 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/d4488205/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #552
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/552/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins335262498959897360.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5189562535030151712.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6833657577611082180.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1646882316713972569.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins601544574578761039.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-12 18:00:38,630 f7b7b85e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f7b7b85e/pkb.log
2017-11-12 18:00:38,630 f7b7b85e MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-12 18:00:38,631 f7b7b85e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-12 18:00:38,904 f7b7b85e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-12 18:00:38,904 f7b7b85e MainThread INFO Initializing the edw service decoder
2017-11-12 18:00:39,053 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-12 18:00:39,056 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-12 18:00:39,057 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-12 18:00:44,804 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-12 18:14:49,823 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-12 18:14:49,835 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f7b7b85e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-12 18:14:50,099 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f7b7b85e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/f7b7b85e/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-12 18:14:50,099 f7b7b85e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-12 18:14:50,100 f7b7b85e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-12 18:14:50,102 f7b7b85e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-12 18:14:50,150 f7b7b85e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-12 18:14:50,150 f7b7b85e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f7b7b85e/pkb.log
2017-11-12 18:14:50,151 f7b7b85e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f7b7b85e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #551
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/551/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5479878812116582075.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7374878388638564174.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1194374043947575490.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6654671326371034233.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1555278333558075064.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-12 12:00:13,464 f1a04ea4 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f1a04ea4/pkb.log
2017-11-12 12:00:13,465 f1a04ea4 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-12 12:00:13,465 f1a04ea4 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-12 12:00:13,615 f1a04ea4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-12 12:00:13,616 f1a04ea4 MainThread INFO Initializing the edw service decoder
2017-11-12 12:00:13,769 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-12 12:00:13,772 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-12 12:00:13,773 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-12 12:00:19,900 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-12 12:13:18,204 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-12 12:13:18,215 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f1a04ea4/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-12 12:13:18,360 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f1a04ea4/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/f1a04ea4/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-12 12:13:18,360 f1a04ea4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-12 12:13:18,361 f1a04ea4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-12 12:13:18,363 f1a04ea4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-12 12:13:18,415 f1a04ea4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-12 12:13:18,416 f1a04ea4 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f1a04ea4/pkb.log
2017-11-12 12:13:18,416 f1a04ea4 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f1a04ea4/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #550
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/550/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8081816558434184159.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7642854607035897007.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6618019841765986839.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2786560654335606097.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4057440922930085373.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-12 06:00:21,920 868ffde5 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/868ffde5/pkb.log
2017-11-12 06:00:21,920 868ffde5 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-12 06:00:21,921 868ffde5 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-12 06:00:22,160 868ffde5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-12 06:00:22,161 868ffde5 MainThread INFO Initializing the edw service decoder
2017-11-12 06:00:22,340 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-12 06:00:22,343 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-12 06:00:22,343 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-12 06:00:29,368 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-12 06:14:21,613 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-12 06:14:21,625 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/868ffde5/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-12 06:14:21,909 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/868ffde5/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/868ffde5/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-12 06:14:21,909 868ffde5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-12 06:14:21,910 868ffde5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-12 06:14:21,912 868ffde5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-12 06:14:21,965 868ffde5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-12 06:14:21,966 868ffde5 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/868ffde5/pkb.log
2017-11-12 06:14:21,966 868ffde5 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/868ffde5/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #549
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/549/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1082247743103873613.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins833419154024725005.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins530205589369013747.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2550043399002528072.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins225169490369052101.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-12 00:00:26,056 1dfe0de8 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/1dfe0de8/pkb.log
2017-11-12 00:00:26,056 1dfe0de8 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-12 00:00:26,056 1dfe0de8 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-12 00:00:26,309 1dfe0de8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-12 00:00:26,310 1dfe0de8 MainThread INFO Initializing the edw service decoder
2017-11-12 00:00:26,465 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-12 00:00:26,468 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-12 00:00:26,468 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-12 00:00:32,147 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-12 00:15:47,269 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-12 00:15:47,282 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1dfe0de8/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-12 00:15:47,431 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1dfe0de8/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/1dfe0de8/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-12 00:15:47,432 1dfe0de8 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-12 00:15:47,433 1dfe0de8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-12 00:15:47,435 1dfe0de8 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-12 00:15:47,458 1dfe0de8 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-12 00:15:47,459 1dfe0de8 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/1dfe0de8/pkb.log
2017-11-12 00:15:47,459 1dfe0de8 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/1dfe0de8/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #548
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/548/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3819797062300009477.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4787976084026769490.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1642138068874123379.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5896364088509652725.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3439266996375919912.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-11 18:00:16,679 22f67d82 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/22f67d82/pkb.log
2017-11-11 18:00:16,679 22f67d82 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-11 18:00:16,680 22f67d82 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-11 18:00:17,003 22f67d82 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-11 18:00:17,003 22f67d82 MainThread INFO Initializing the edw service decoder
2017-11-11 18:00:17,197 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-11 18:00:17,199 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-11 18:00:17,200 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-11 18:00:25,487 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-11 18:13:32,314 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-11 18:13:32,325 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/22f67d82/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-11 18:13:32,474 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/22f67d82/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/22f67d82/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-11 18:13:32,475 22f67d82 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-11 18:13:32,476 22f67d82 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-11 18:13:32,477 22f67d82 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-11 18:13:32,528 22f67d82 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-11 18:13:32,529 22f67d82 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/22f67d82/pkb.log
2017-11-11 18:13:32,529 22f67d82 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/22f67d82/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #547
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/547/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision acfab89656d1af42e664b9b86942c9848f29f477 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f acfab89656d1af42e664b9b86942c9848f29f477
Commit message: "This closes #4081"
> git rev-list acfab89656d1af42e664b9b86942c9848f29f477 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2840861466951031677.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3505377913873532588.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3057114995979062516.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8039225303408257631.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6642609308400624639.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-11 12:00:22,777 d3452c10 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/d3452c10/pkb.log
2017-11-11 12:00:22,778 d3452c10 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-11 12:00:22,778 d3452c10 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-11 12:00:23,028 d3452c10 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-11 12:00:23,029 d3452c10 MainThread INFO Initializing the edw service decoder
2017-11-11 12:00:23,212 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-11 12:00:23,216 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-11 12:00:23,217 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-11 12:00:30,250 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-11 12:14:28,006 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-11 12:14:28,017 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/d3452c10/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-11 12:14:28,166 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/d3452c10/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/d3452c10/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-11 12:14:28,167 d3452c10 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-11 12:14:28,168 d3452c10 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-11 12:14:28,170 d3452c10 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-11 12:14:28,181 d3452c10 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-11 12:14:28,181 d3452c10 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/d3452c10/pkb.log
2017-11-11 12:14:28,182 d3452c10 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/d3452c10/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)