You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/11/15 12:13:19 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #561
See <https://builds.apache.org/job/beam_PerformanceTests_Python/561/display/redirect?page=changes>
Changes:
[chamikara] Updated jdbc's pom.xml
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7ce0a82b06c29247713c009d32b53fe53fd7ec6d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7ce0a82b06c29247713c009d32b53fe53fd7ec6d
Commit message: "This closes #4121"
> git rev-list 209bb6d748bb776cdb5d93729144ba34dd9cb967 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7582666607470916224.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6778610571247056304.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5383684519806896547.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7107198820040444271.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6243242900567046194.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-15 12:00:22,491 fc892df9 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/fc892df9/pkb.log
2017-11-15 12:00:22,491 fc892df9 MainThread INFO PerfKitBenchmarker version: v1.12.0-183-g427999d
2017-11-15 12:00:22,492 fc892df9 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-15 12:00:22,917 fc892df9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-15 12:00:22,917 fc892df9 MainThread INFO Initializing the edw service decoder
2017-11-15 12:00:23,227 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-15 12:00:23,232 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-15 12:00:23,233 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-15 12:00:36,717 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-15 12:12:56,301 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-15 12:12:56,320 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/fc892df9/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-15 12:12:56,714 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/fc892df9/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/fc892df9/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-15 12:12:56,714 fc892df9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 582, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 493, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-15 12:12:56,715 fc892df9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-15 12:12:56,718 fc892df9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-15 12:12:56,766 fc892df9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-15 12:12:56,767 fc892df9 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/fc892df9/pkb.log
2017-11-15 12:12:56,767 fc892df9 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/fc892df9/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user herohde@google.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Jenkins build is back to normal : beam_PerformanceTests_Python #586
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/586/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_Python #585
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/585/display/redirect?page=changes>
Changes:
[robertwb] Add logging and test aborting for timeouts.
[lcwik] Add precommit integration tests to examples/java
[lcwik] Update Gradle PreCommit to publish test results.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 603f86a26122aa611d677e121485ff835c6662d1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 603f86a26122aa611d677e121485ff835c6662d1
Commit message: "Add precommit integration tests to examples/java"
> git rev-list fb41b2950fa2f59f5ff7496af6990c66a715910e # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2008934987287236260.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins964574941314236944.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3760054871174153221.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1422507380351186321.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7184233450391297771.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-22 00:00:14,976 6b318240 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6b318240/pkb.log
2017-11-22 00:00:14,976 6b318240 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-22 00:00:14,976 6b318240 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-22 00:00:15,227 6b318240 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-22 00:00:15,227 6b318240 MainThread INFO Initializing the edw service decoder
2017-11-22 00:00:15,377 6b318240 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-22 00:00:15,380 6b318240 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-22 00:00:15,381 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-22 00:00:22,620 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-22 00:06:26,903 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-22 00:06:26,913 6b318240 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-22 00:16:26,917 6b318240 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-22 00:16:26,927 6b318240 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/6b318240/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-22 00:16:26,928 6b318240 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-22 00:16:26,929 6b318240 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-22 00:16:26,931 6b318240 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-22 00:16:26,945 6b318240 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-22 00:16:26,946 6b318240 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6b318240/pkb.log
2017-11-22 00:16:26,946 6b318240 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6b318240/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #584
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/584/display/redirect>
------------------------------------------
GitHub pull request #4158 of commit c8a5187cd2850613d2881909363e32b19d1155c3, no merge conflicts.
Setting status of c8a5187cd2850613d2881909363e32b19d1155c3 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/584/ and message: 'Build started sha1 is merged.'
Using context: Jenkins: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/4158/*:refs/remotes/origin/pr/4158/*
> git rev-parse refs/remotes/origin/pr/4158/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/4158/merge^{commit} # timeout=10
Checking out Revision c4d3f44854ff8537dea2121622d73aa31bdd8c88 (refs/remotes/origin/pr/4158/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f c4d3f44854ff8537dea2121622d73aa31bdd8c88
Commit message: "Merge c8a5187cd2850613d2881909363e32b19d1155c3 into e9d746a34a82791571b002a78c62e9f46efd4253"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5993992038028145613.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2050962319129494155.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins306321548366558932.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8931499395194623189.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5579537586103628955.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 23:41:09,835 6b83eb7e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6b83eb7e/pkb.log
2017-11-21 23:41:09,836 6b83eb7e MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 23:41:09,837 6b83eb7e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 23:41:10,458 6b83eb7e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 23:41:10,469 6b83eb7e MainThread INFO Initializing the edw service decoder
2017-11-21 23:41:10,967 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 23:41:10,981 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 23:41:10,982 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 23:41:23,047 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 23:48:48,571 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 23:48:48,581 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 23:58:48,586 6b83eb7e Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-21 23:58:48,590 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/6b83eb7e/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-21 23:58:48,591 6b83eb7e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 23:58:48,592 6b83eb7e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 23:58:48,594 6b83eb7e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 23:58:48,617 6b83eb7e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 23:58:48,617 6b83eb7e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6b83eb7e/pkb.log
2017-11-21 23:58:48,617 6b83eb7e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6b83eb7e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #583
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/583/display/redirect?page=changes>
Changes:
[robertwb] Add option to run PTransform steps from PipelineRunner
[robertwb] Minor cleanup.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision fb41b2950fa2f59f5ff7496af6990c66a715910e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f fb41b2950fa2f59f5ff7496af6990c66a715910e
Commit message: "Closes #4123"
> git rev-list ed40093d8bf47d23e7215d177415f26bfba83fea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7240721229410407952.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2333467120525341955.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2211792954215740008.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7263338426728025283.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1821733305915587363.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 18:27:08,967 e3c22b77 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/e3c22b77/pkb.log
2017-11-21 18:27:08,968 e3c22b77 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 18:27:08,969 e3c22b77 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 18:27:09,344 e3c22b77 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 18:27:09,345 e3c22b77 MainThread INFO Initializing the edw service decoder
2017-11-21 18:27:09,571 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 18:27:09,575 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 18:27:09,576 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 18:27:21,239 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 18:43:35,080 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 18:43:35,090 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/e3c22b77/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 18:43:35,421 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/e3c22b77/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/e3c22b77/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-21 18:43:35,422 e3c22b77 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 18:43:35,437 e3c22b77 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 18:43:35,439 e3c22b77 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 18:43:35,458 e3c22b77 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 18:43:35,459 e3c22b77 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/e3c22b77/pkb.log
2017-11-21 18:43:35,459 e3c22b77 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/e3c22b77/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #582
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/582/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed40093d8bf47d23e7215d177415f26bfba83fea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed40093d8bf47d23e7215d177415f26bfba83fea
Commit message: "Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits."
> git rev-list ed40093d8bf47d23e7215d177415f26bfba83fea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5597192447227310593.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2750295580613019511.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4752303249881969147.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1929723363690529870.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7537600052014652448.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 12:00:18,094 0256554e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0256554e/pkb.log
2017-11-21 12:00:18,095 0256554e MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 12:00:18,095 0256554e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 12:00:18,337 0256554e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 12:00:18,338 0256554e MainThread INFO Initializing the edw service decoder
2017-11-21 12:00:18,516 0256554e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 12:00:18,519 0256554e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 12:00:18,519 0256554e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 12:00:24,998 0256554e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 12:06:40,615 0256554e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 12:06:40,627 0256554e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 12:16:40,632 0256554e Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-21 12:16:40,639 0256554e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/0256554e/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-21 12:16:40,639 0256554e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 12:16:40,640 0256554e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 12:16:40,642 0256554e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 12:16:40,696 0256554e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 12:16:40,697 0256554e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0256554e/pkb.log
2017-11-21 12:16:40,698 0256554e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0256554e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #581
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/581/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed40093d8bf47d23e7215d177415f26bfba83fea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed40093d8bf47d23e7215d177415f26bfba83fea
Commit message: "Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits."
> git rev-list ed40093d8bf47d23e7215d177415f26bfba83fea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5133738960247356081.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5050038049388389462.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2667335010300677326.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6647271871107147993.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3424496056567103648.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-21 06:00:26,591 ca162ee3 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ca162ee3/pkb.log
2017-11-21 06:00:26,592 ca162ee3 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-21 06:00:26,592 ca162ee3 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-21 06:00:26,884 ca162ee3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-21 06:00:26,884 ca162ee3 MainThread INFO Initializing the edw service decoder
2017-11-21 06:00:27,076 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-21 06:00:27,080 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-21 06:00:27,081 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-21 06:00:33,554 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-21 06:07:48,128 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-21 06:07:48,139 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-21 06:17:48,144 ca162ee3 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-21 06:17:48,149 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/ca162ee3/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-21 06:17:48,150 ca162ee3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-21 06:17:48,150 ca162ee3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-21 06:17:48,152 ca162ee3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-21 06:17:48,184 ca162ee3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-21 06:17:48,185 ca162ee3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ca162ee3/pkb.log
2017-11-21 06:17:48,185 ca162ee3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ca162ee3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #580
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/580/display/redirect?page=changes>
Changes:
[lcwik] Enable Java Gradle Precommit in Jenkins
[lcwik] Simplify maintainability by generating tasks and command line from list
[lcwik] Limit the commit status length to 255 characters to honor Github limit
[lcwik] Revert PR/3976, the Jenkins Gradle plugin expects the Gradle Wrapper to
[lcwik] Sync with master and add new subproject for beam-runners-local-java-core
[kirpichov] Improved validation of DynamicDestinations results in BigQueryIO.write
[chamikara] Dataflow and PerfKit profiles; hash for 100.000.000 lines
[lcwik] Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other
[lcwik] Update the location of the root build directory to coincide with the
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed40093d8bf47d23e7215d177415f26bfba83fea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed40093d8bf47d23e7215d177415f26bfba83fea
Commit message: "Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other precommits."
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5230896743662012013.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8667876498156484861.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1337895185962056481.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8231665045133042800.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
sdks/python/[gcp,test] should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #579
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/579/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5373597544739417499.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5592195336555117248.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3763933392533803781.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4999056484096096630.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1784607244369668976.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-20 12:00:23,001 0ac4309d MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0ac4309d/pkb.log
2017-11-20 12:00:23,001 0ac4309d MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-20 12:00:23,002 0ac4309d MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-20 12:00:23,200 0ac4309d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-20 12:00:23,201 0ac4309d MainThread INFO Initializing the edw service decoder
2017-11-20 12:00:23,356 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-20 12:00:23,359 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-20 12:00:23,359 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-20 12:00:30,653 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-20 12:06:35,077 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-20 12:06:35,088 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-20 12:16:35,095 0ac4309d Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-20 12:16:35,099 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/0ac4309d/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-20 12:16:35,100 0ac4309d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-20 12:16:35,102 0ac4309d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-20 12:16:35,104 0ac4309d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-20 12:16:35,154 0ac4309d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-20 12:16:35,155 0ac4309d MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0ac4309d/pkb.log
2017-11-20 12:16:35,155 0ac4309d MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0ac4309d/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #578
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/578/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins767942611611723038.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4174273406739557118.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins761864003373348100.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5839146230927018378.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1249975475942734405.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-20 06:00:16,653 f5729124 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f5729124/pkb.log
2017-11-20 06:00:16,653 f5729124 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-20 06:00:16,653 f5729124 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-20 06:00:16,781 f5729124 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-20 06:00:16,782 f5729124 MainThread INFO Initializing the edw service decoder
2017-11-20 06:00:16,951 f5729124 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-20 06:00:16,954 f5729124 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-20 06:00:16,954 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-20 06:00:23,564 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-20 06:06:41,707 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-20 06:06:41,718 f5729124 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-20 06:16:41,723 f5729124 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-20 06:16:41,730 f5729124 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/f5729124/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-20 06:16:41,730 f5729124 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-20 06:16:41,731 f5729124 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-20 06:16:41,733 f5729124 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-20 06:16:41,759 f5729124 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-20 06:16:41,760 f5729124 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f5729124/pkb.log
2017-11-20 06:16:41,760 f5729124 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f5729124/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #577
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/577/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5014233780848433318.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3041297463291230105.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2773657514736843752.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1538629637840030819.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7506920532129854367.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-20 00:00:25,442 69fe8637 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/69fe8637/pkb.log
2017-11-20 00:00:25,442 69fe8637 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-20 00:00:25,443 69fe8637 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-20 00:00:25,828 69fe8637 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-20 00:00:25,828 69fe8637 MainThread INFO Initializing the edw service decoder
2017-11-20 00:00:26,016 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-20 00:00:26,019 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-20 00:00:26,020 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-20 00:00:31,550 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-20 00:07:44,579 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-20 00:07:44,589 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-20 00:17:44,594 69fe8637 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-20 00:17:44,600 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/69fe8637/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-20 00:17:44,600 69fe8637 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-20 00:17:44,601 69fe8637 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-20 00:17:44,603 69fe8637 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-20 00:17:44,636 69fe8637 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-20 00:17:44,637 69fe8637 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/69fe8637/pkb.log
2017-11-20 00:17:44,637 69fe8637 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/69fe8637/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #576
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/576/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5520766847453943497.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6226155290360226877.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4145240403247521989.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7971707459832702207.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7480621607496451934.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 18:00:17,317 85b96e9a MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/85b96e9a/pkb.log
2017-11-19 18:00:17,317 85b96e9a MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 18:00:17,318 85b96e9a MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 18:00:17,559 85b96e9a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 18:00:17,560 85b96e9a MainThread INFO Initializing the edw service decoder
2017-11-19 18:00:17,712 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 18:00:17,715 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 18:00:17,715 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 18:00:24,552 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 18:06:30,931 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 18:06:30,942 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 18:16:30,947 85b96e9a Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-19 18:16:30,954 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/85b96e9a/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-19 18:16:30,955 85b96e9a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 18:16:30,956 85b96e9a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 18:16:30,958 85b96e9a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 18:16:30,985 85b96e9a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 18:16:30,985 85b96e9a MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/85b96e9a/pkb.log
2017-11-19 18:16:30,986 85b96e9a MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/85b96e9a/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #575
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/575/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1471960231046679688.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7811412306516364141.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9167470683863290579.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3869762181712991063.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins44924488039484535.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 12:00:16,109 59025a31 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/59025a31/pkb.log
2017-11-19 12:00:16,109 59025a31 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 12:00:16,110 59025a31 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 12:00:16,324 59025a31 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 12:00:16,324 59025a31 MainThread INFO Initializing the edw service decoder
2017-11-19 12:00:16,497 59025a31 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 12:00:16,501 59025a31 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 12:00:16,502 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 12:00:23,673 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 12:06:29,682 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 12:06:29,693 59025a31 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 12:16:29,698 59025a31 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-19 12:16:29,704 59025a31 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/59025a31/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-19 12:16:29,705 59025a31 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 12:16:29,706 59025a31 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 12:16:29,707 59025a31 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 12:16:29,759 59025a31 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 12:16:29,760 59025a31 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/59025a31/pkb.log
2017-11-19 12:16:29,760 59025a31 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/59025a31/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #574
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/574/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins32354818368900436.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2823221348356969534.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5300710995425967786.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4735169387755430985.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6203202532533460797.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 06:00:27,531 bfd449c6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/bfd449c6/pkb.log
2017-11-19 06:00:27,532 bfd449c6 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 06:00:27,533 bfd449c6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 06:00:28,241 bfd449c6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 06:00:28,241 bfd449c6 MainThread INFO Initializing the edw service decoder
2017-11-19 06:00:28,729 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 06:00:28,733 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 06:00:28,734 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 06:00:46,411 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 06:17:49,609 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 06:17:49,716 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/bfd449c6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 06:17:50,311 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/bfd449c6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/bfd449c6/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-19 06:17:50,312 bfd449c6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 06:17:50,313 bfd449c6 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 06:17:50,315 bfd449c6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 06:17:50,341 bfd449c6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 06:17:50,342 bfd449c6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/bfd449c6/pkb.log
2017-11-19 06:17:50,342 bfd449c6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/bfd449c6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #573
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/573/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6432721446071832801.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1749592125091995284.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3868056377492935426.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins278161475843902295.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.3.0.dev)
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Collecting futures<4.0.0,>=3.1.1 (from apache-beam==2.3.0.dev)
Using cached futures-3.1.1-py2-none-any.whl
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: pyvcf, futures, apache-beam
Found existing installation: futures 3.0.5
Uninstalling futures-3.0.5:
Successfully uninstalled futures-3.0.5
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam futures-3.1.1 pyvcf-0.6.8
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2716840393397482088.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-19 00:01:00,693 a435512b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/a435512b/pkb.log
2017-11-19 00:01:00,693 a435512b MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-19 00:01:00,694 a435512b MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-19 00:01:01,137 a435512b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-19 00:01:01,137 a435512b MainThread INFO Initializing the edw service decoder
2017-11-19 00:01:01,329 a435512b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-19 00:01:01,332 a435512b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-19 00:01:01,333 a435512b MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-19 00:01:07,635 a435512b MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-19 00:08:20,425 a435512b MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-19 00:08:20,437 a435512b MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-19 00:18:20,442 a435512b Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz".
2017-11-19 00:18:20,446 a435512b MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/apache_beam/io/gcp/gcsio.py:122: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/a435512b/beam/sdks/python/apache_beam/coders/typecoders.py:134: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2017-11-19 00:18:20,447 a435512b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-19 00:18:20,448 a435512b MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-19 00:18:20,450 a435512b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-19 00:18:20,473 a435512b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-19 00:18:20,473 a435512b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/a435512b/pkb.log
2017-11-19 00:18:20,474 a435512b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/a435512b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #572
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/572/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6067668881080642432.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8421980976207251306.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6665273338505486915.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1590361676075629645.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 521, in _prepare_file
req_to_install.check_if_exists()
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 1036, in check_if_exists
self.req.name
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 558, in get_distribution
dist = get_provider(dist)
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 432, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 968, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 859, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
ContextualVersionConflict: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures<4.0.0,>=3.1.1'), set(['apache-beam']))
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #571
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/571/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list 022e74460433e0ad9b44356229151c0d7b0af351 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9111897738739701955.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2255391586014356900.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4858012336430098801.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9116202591903373260.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4006141163560924042.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-18 12:00:16,779 18bee801 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/18bee801/pkb.log
2017-11-18 12:00:16,779 18bee801 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-18 12:00:16,780 18bee801 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-18 12:00:16,982 18bee801 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-18 12:00:16,982 18bee801 MainThread INFO Initializing the edw service decoder
2017-11-18 12:00:17,135 18bee801 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-18 12:00:17,138 18bee801 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-18 12:00:17,138 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-18 12:00:24,065 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-18 12:06:43,199 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-18 12:06:43,209 18bee801 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/18bee801/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-18 12:06:43,352 18bee801 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/18bee801/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/18bee801/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-18 12:06:43,353 18bee801 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-18 12:06:43,354 18bee801 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-18 12:06:43,355 18bee801 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-18 12:06:43,393 18bee801 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-18 12:06:43,394 18bee801 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/18bee801/pkb.log
2017-11-18 12:06:43,394 18bee801 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/18bee801/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #570
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/570/display/redirect?page=changes>
Changes:
[kedin] [Nexmark] Extract GeneratorCheckpoint into a separate class. Move
[kedin] [Nexmark] Extract PersonGenerator, StringsGenerator, LongGenerator from
[kedin] [Nexmark] Extract AuctionGenerator, PriceGenerator from Generator
[kedin] [Nexmark] Extract BidGenerator from Generator
[lcwik] Move AVRO files to avro source location
[lcwik] Initial gradle build definitions for all java based modules which is
[lcwik] Add license headers to all build.gradle files.
[lcwik] Add provided scope plugin and migrate compileOnly/testCompileOnly to
[lcwik] Add the ability to list dependencies recursively. Also ban the
[lcwik] Update comments
[lcwik] Configure the checkstyle plugin
[lcwik] Add support to run the Apache Rat plugin Add gradle generated files to
[lcwik] Adding Python gradle changes
[lcwik] Fix python tasks to not re-use the default gradle lifecycle names.
[lcwik] Add test task dependencies for testCython and testGcp and also add lint
[lcwik] Create build_rules file moving out specific plugins to them.
[lcwik] Add missing build_rules file.
[lcwik] Ignore test categories NeedsRunner, ValidatesRunner, and all *IT tests
[lcwik] Add code style validation and auto-formatting to Gradle build.
[lcwik] Add beam-sdks-java-core dependency to beam-local-artifact-service
[lcwik] Add gradle build for sdks/go
[lcwik] Get evaluation dependency order fixed because of dependency on
[lcwik] Update comments on root project
[lcwik] Set beamUseDummyRunner = true
[lcwik] Move spotless plugin to delegate config
[lcwik] Fix spotless config
[lcwik] Change hamcrest and junit to testCompile in
[lcwik] Add findbugs to the build
[lcwik] Add test to check phase
[lcwik] Filter resources to set dataflow environrment versions in
[lcwik] Force dependency resolution of common Java dependencies
[lcwik] Filter sdk.properties, set pom.version and build timestamp
[lcwik] Use dummy runner in tests in google-cloud-dataflow-java
[lcwik] Add test dependecy on beam-sdks-java-core to
[lcwik] Migrate findbugs to be configurable from the JavaNatureConfiguration
[lcwik] Setup several maven-surefire-plugin configurations
[lcwik] Remove added sdk-core dependencies because of
[lcwik] Fix provided scope in sdks/java/extensions/sorter
[lcwik] Ban hamcrest-all/mockito-all and migrate to hamcrest-core/mockito-core
[lcwik] Add gradle rule for python container
[lcwik] Fix python container gradle build
[lcwik] Fix hadoop-file-system test dependencies Explain why hcatalog is failing
[lcwik] Add license header to sdks/python/build.gradle
[lcwik] Get spark runner unit tests to pass.
[lcwik] Define the correct versions of apex-core Add comments to hcatalog/hbase
[lcwik] Simplify ApexYarnLauncher matching Attempt to create dependency tree
[lcwik] Enable configuraton on demand by default Get ApexYarnLauncher test to
[lcwik] Fix sdks/java/core testCompile dependency to bring in
[lcwik] Re-enable Python SDK
[lcwik] Disable findbugs within sdks/java/io/kafka to match findbugs
[lcwik] Add a task dependency visualization plugin. Resolve a spurious error
[lcwik] Fix classpath bundling issue caused by Hive and its dependencies
[lcwik] Define Tika dependencies version as variable
[lcwik] Rebase on apache/master and add new subprojects, update versions of
[lcwik] Skip test failures in hadoop related modules due to dependency
[lcwik] Add Jacoco plugin
[lcwik] Add Gradle builds for gcemd and gcsproxy
[lcwik] Add Java SDK Container Gradle build file
[lcwik] Create uber jar of the Java SDK harness and copy it into the location
[lcwik] Make docker tasks correctly depend on required build tasks
[lcwik] Fix runners/gcp/* go package paths
[lcwik] Add shadow configuration for sdks/java/core updating all other projects
[lcwik] Configure default shading profile on all java sub-projects to shade away
[lcwik] Shade direct-java and sql projects. Update repackaged locations to
[lcwik] Fix gcemd and gcsproxy builds. Fix whitespace formatting.
[lcwik] Enforce that gogradle dependency resolution/installation is done
[lcwik] Also ensure that resolving test dependencies are done sequentially with
[lcwik] Use valid repackage path on shadow relocations Fix Spark runners tests
[lcwik] Move user.avsc to directory which corresponds with namespace under
[lcwik] Add comments to various parts of build_rules.gradle
[lcwik] Fix excludes to correctly match the .idea/ directory.
[lcwik] Update build.gradle files to correspond with pom.xml changes within
[lcwik] Enable FFMP -> JavaCC path to generate Beam SQL parser for
[tgroh] Add a Local Java Core Module
[tgroh] Move StructuralKey to the local execution module
[tgroh] Add A TransformExecutorFactory
[chamikara] Clarify documentation on support for reading from/writing to time
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 022e74460433e0ad9b44356229151c0d7b0af351 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 022e74460433e0ad9b44356229151c0d7b0af351
Commit message: "This closes #4147"
> git rev-list f10399d7c374d0342fdac524c8a2ddf4d7ec5cd3 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2133352925076915801.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7168219796139475913.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins988525984695506756.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins544213913297504256.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7832708401297371404.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-18 06:00:32,127 73ec6fc3 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/73ec6fc3/pkb.log
2017-11-18 06:00:32,127 73ec6fc3 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-18 06:00:32,128 73ec6fc3 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-18 06:00:32,379 73ec6fc3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-18 06:00:32,379 73ec6fc3 MainThread INFO Initializing the edw service decoder
2017-11-18 06:00:32,561 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-18 06:00:32,564 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-18 06:00:32,565 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-18 06:00:39,075 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-18 06:08:21,383 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-18 06:08:21,397 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/73ec6fc3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-18 06:08:21,700 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/73ec6fc3/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/73ec6fc3/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-18 06:08:21,701 73ec6fc3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-18 06:08:21,702 73ec6fc3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-18 06:08:21,703 73ec6fc3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-18 06:08:21,749 73ec6fc3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-18 06:08:21,750 73ec6fc3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/73ec6fc3/pkb.log
2017-11-18 06:08:21,750 73ec6fc3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/73ec6fc3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user kedin@kedin-macbookpro.roam.corp.google.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #569
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/569/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3206] Shut down executor when spark runner finishes
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f10399d7c374d0342fdac524c8a2ddf4d7ec5cd3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f10399d7c374d0342fdac524c8a2ddf4d7ec5cd3
Commit message: "This closes #4138"
> git rev-list 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins343152140608123410.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins88728062273622039.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6844791055540984298.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3694502064599885753.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7140621039230471732.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 18:00:26,432 714bba78 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/714bba78/pkb.log
2017-11-17 18:00:26,433 714bba78 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 18:00:26,434 714bba78 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 18:00:27,042 714bba78 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 18:00:27,043 714bba78 MainThread INFO Initializing the edw service decoder
2017-11-17 18:00:27,266 714bba78 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 18:00:27,270 714bba78 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 18:00:27,271 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 18:00:40,271 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 18:18:00,529 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 18:18:00,552 714bba78 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/714bba78/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 18:18:00,872 714bba78 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/714bba78/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/714bba78/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 18:18:00,873 714bba78 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 18:18:00,874 714bba78 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 18:18:00,877 714bba78 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 18:18:00,891 714bba78 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 18:18:00,891 714bba78 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/714bba78/pkb.log
2017-11-17 18:18:00,891 714bba78 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/714bba78/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #568
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/568/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee
Commit message: "This closes #4057: [BEAM-3035] Extract ReifyTimestampsAndWindows"
> git rev-list 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1769809359256019078.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3303174299824323028.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1771786221137633440.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins642950221683200409.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7981304723880363437.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 12:00:29,475 5fdb6c75 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/5fdb6c75/pkb.log
2017-11-17 12:00:29,476 5fdb6c75 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 12:00:29,477 5fdb6c75 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 12:00:29,956 5fdb6c75 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 12:00:29,956 5fdb6c75 MainThread INFO Initializing the edw service decoder
2017-11-17 12:00:30,326 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 12:00:30,330 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 12:00:30,331 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 12:00:42,818 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 12:13:14,706 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 12:13:14,720 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5fdb6c75/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 12:13:14,974 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5fdb6c75/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/5fdb6c75/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 12:13:14,975 5fdb6c75 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 12:13:14,976 5fdb6c75 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 12:13:14,978 5fdb6c75 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 12:13:15,008 5fdb6c75 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 12:13:15,008 5fdb6c75 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/5fdb6c75/pkb.log
2017-11-17 12:13:15,008 5fdb6c75 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/5fdb6c75/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #567
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/567/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee
Commit message: "This closes #4057: [BEAM-3035] Extract ReifyTimestampsAndWindows"
> git rev-list 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5322896415489438698.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8955443008791098663.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5560883738286550035.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4209878166945532977.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4439886988845878833.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 06:00:19,288 0fc1b167 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0fc1b167/pkb.log
2017-11-17 06:00:19,288 0fc1b167 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 06:00:19,289 0fc1b167 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 06:00:19,748 0fc1b167 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 06:00:19,749 0fc1b167 MainThread INFO Initializing the edw service decoder
2017-11-17 06:00:19,974 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 06:00:19,978 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 06:00:19,979 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 06:00:27,897 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 06:11:11,083 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 06:11:11,118 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0fc1b167/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 06:11:11,466 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0fc1b167/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/0fc1b167/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 06:11:11,466 0fc1b167 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 06:11:11,515 0fc1b167 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 06:11:11,518 0fc1b167 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 06:11:11,555 0fc1b167 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 06:11:11,556 0fc1b167 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0fc1b167/pkb.log
2017-11-17 06:11:11,556 0fc1b167 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0fc1b167/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #566
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/566/display/redirect?page=changes>
Changes:
[valentyn] Fix user_score example.
[lcwik] [BEAM-3202] Ensure that PipelineOptions.getOptionsId is always
[kirpichov] [BEAM-3035] Introduces Reify transform
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0df7ba9d6791bf06a59e3e7d0cb9500bbaa2b1ee
Commit message: "This closes #4057: [BEAM-3035] Extract ReifyTimestampsAndWindows"
> git rev-list 92013ec9e659228dd2ea66a28e2d83e7f4795b61 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9021074132772576932.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1454103446174442545.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5973398805072968925.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4032666837727789652.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7329995538676718832.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-17 00:00:37,168 b3bd6d5f MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b3bd6d5f/pkb.log
2017-11-17 00:00:37,169 b3bd6d5f MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-17 00:00:37,170 b3bd6d5f MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-17 00:00:37,568 b3bd6d5f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-17 00:00:37,569 b3bd6d5f MainThread INFO Initializing the edw service decoder
2017-11-17 00:00:37,884 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-17 00:00:37,889 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-17 00:00:37,890 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-17 00:00:50,820 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-17 00:13:50,350 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-17 00:13:50,366 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b3bd6d5f/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-17 00:13:50,784 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b3bd6d5f/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/b3bd6d5f/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-17 00:13:50,784 b3bd6d5f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-17 00:13:50,785 b3bd6d5f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-17 00:13:50,788 b3bd6d5f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-17 00:13:50,798 b3bd6d5f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-17 00:13:50,798 b3bd6d5f MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b3bd6d5f/pkb.log
2017-11-17 00:13:50,798 b3bd6d5f MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b3bd6d5f/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #565
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/565/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92013ec9e659228dd2ea66a28e2d83e7f4795b61 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92013ec9e659228dd2ea66a28e2d83e7f4795b61
Commit message: "This closes #4137: [BEAM-3195] Specifies numShards on windowed writes examples, as it is now required"
> git rev-list 92013ec9e659228dd2ea66a28e2d83e7f4795b61 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3121235113020595809.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1648897548745660183.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4103012481848862924.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2433649201481481848.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8493788332974716657.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 18:00:25,410 8954b74b MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/8954b74b/pkb.log
2017-11-16 18:00:25,410 8954b74b MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 18:00:25,411 8954b74b MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 18:00:25,863 8954b74b MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 18:00:25,864 8954b74b MainThread INFO Initializing the edw service decoder
2017-11-16 18:00:26,050 8954b74b MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 18:00:26,053 8954b74b MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 18:00:26,054 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 18:00:34,157 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 18:11:05,310 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 18:11:05,326 8954b74b MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8954b74b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 18:11:05,640 8954b74b MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8954b74b/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/8954b74b/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 18:11:05,640 8954b74b MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 18:11:05,641 8954b74b MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 18:11:05,644 8954b74b MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 18:11:05,695 8954b74b MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 18:11:05,696 8954b74b MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/8954b74b/pkb.log
2017-11-16 18:11:05,696 8954b74b MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/8954b74b/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #564
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/564/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92013ec9e659228dd2ea66a28e2d83e7f4795b61 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92013ec9e659228dd2ea66a28e2d83e7f4795b61
Commit message: "This closes #4137: [BEAM-3195] Specifies numShards on windowed writes examples, as it is now required"
> git rev-list 92013ec9e659228dd2ea66a28e2d83e7f4795b61 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins942289546882004076.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3611155006219490356.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3822713513701605842.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1001870562125754694.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4850996491855444649.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 12:00:22,634 81071ae6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/81071ae6/pkb.log
2017-11-16 12:00:22,635 81071ae6 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 12:00:22,636 81071ae6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 12:00:23,059 81071ae6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 12:00:23,059 81071ae6 MainThread INFO Initializing the edw service decoder
2017-11-16 12:00:23,325 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 12:00:23,329 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 12:00:23,330 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 12:00:36,147 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 12:12:37,351 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 12:12:37,392 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/81071ae6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 12:12:37,712 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/81071ae6/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/81071ae6/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 12:12:37,712 81071ae6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 12:12:37,713 81071ae6 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 12:12:37,716 81071ae6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 12:12:37,733 81071ae6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 12:12:37,733 81071ae6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/81071ae6/pkb.log
2017-11-16 12:12:37,733 81071ae6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/81071ae6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #563
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/563/display/redirect?page=changes>
Changes:
[kirpichov] Specifies numShards on windowed writes examples, as it is now required
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92013ec9e659228dd2ea66a28e2d83e7f4795b61 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92013ec9e659228dd2ea66a28e2d83e7f4795b61
Commit message: "This closes #4137: [BEAM-3195] Specifies numShards on windowed writes examples, as it is now required"
> git rev-list c3a96bf3d9d40f7636d8ac62326aa75e93c0b11c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6193826000221498997.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6867784704250554535.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1664628939435077919.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6317801227600658056.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5743809504303722366.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 06:00:37,736 f1c8efd8 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f1c8efd8/pkb.log
2017-11-16 06:00:37,737 f1c8efd8 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 06:00:37,738 f1c8efd8 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 06:00:38,317 f1c8efd8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 06:00:38,318 f1c8efd8 MainThread INFO Initializing the edw service decoder
2017-11-16 06:00:38,626 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 06:00:38,631 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 06:00:38,631 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 06:00:52,509 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 06:14:45,636 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 06:14:45,661 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f1c8efd8/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 06:14:46,347 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f1c8efd8/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/f1c8efd8/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 06:14:46,348 f1c8efd8 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 06:14:46,358 f1c8efd8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 06:14:46,361 f1c8efd8 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 06:14:46,385 f1c8efd8 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 06:14:46,386 f1c8efd8 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f1c8efd8/pkb.log
2017-11-16 06:14:46,386 f1c8efd8 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f1c8efd8/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Build failed in Jenkins: beam_PerformanceTests_Python #562
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/562/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-2926] Migrate to using a trivial multimap materialization within
[lcwik] Rebase and update code to honor findbugs @Nullable/@Nonnull conversion
[lcwik] Update Dataflow worker image to be compatible with side input changes.
[lcwik] Replace the View.As transforms for Dataflow batch because the entire
[lcwik] Fix BatchViewOverrides ViewAsSingleton to apply the combine fn that was
[altay] Fixed adding timestamp and id attributes to pubsub messages
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision c3a96bf3d9d40f7636d8ac62326aa75e93c0b11c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f c3a96bf3d9d40f7636d8ac62326aa75e93c0b11c
Commit message: "This closes #3581"
> git rev-list 7ce0a82b06c29247713c009d32b53fe53fd7ec6d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8976025187010219755.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4509251861334500213.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6514203397664823258.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4091882124589489934.sh
+ pip install --user -e 'sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: grpcio<2.0,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: oauth2client<4.0.0,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: protobuf<=3.3.0,>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: six<1.11,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-apitools<=0.5.11,>=0.5.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2.0,>=1.0->apache-beam==2.3.0.dev)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<4.0.0,>=2.0.1->apache-beam==2.3.0.dev)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from protobuf<=3.3.0,>=3.2.0->apache-beam==2.3.0.dev)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8378667589799528935.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2017-11-16 00:00:35,522 3ecb3927 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/3ecb3927/pkb.log
2017-11-16 00:00:35,523 3ecb3927 MainThread INFO PerfKitBenchmarker version: v1.12.0-198-g903eb82
2017-11-16 00:00:35,524 3ecb3927 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2017-11-16 00:00:36,231 3ecb3927 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2017-11-16 00:00:36,231 3ecb3927 MainThread INFO Initializing the edw service decoder
2017-11-16 00:00:36,619 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2017-11-16 00:00:36,624 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2017-11-16 00:00:36,625 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2017-11-16 00:00:50,512 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2017-11-16 00:13:49,340 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2017-11-16 00:13:49,360 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3ecb3927/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz
2017-11-16 00:13:49,704 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3ecb3927/beam/sdks/python/target/apache-beam-2.3.0.dev.tar.gz. Got return code (1).
STDOUT:
STDERR: /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
/tmp/perfkitbenchmarker/runs/3ecb3927/beam/sdks/python/nose-1.3.7-py2.7.egg/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: (futures 3.0.5 (/home/jenkins/.local/lib/python2.7/site-packages), Requirement.parse('futures>=3.1.1,<4.0.0'))
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2017-11-16 00:13:49,704 3ecb3927 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 584, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 495, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2017-11-16 00:13:49,705 3ecb3927 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2017-11-16 00:13:49,708 3ecb3927 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2017-11-16 00:13:49,756 3ecb3927 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2017-11-16 00:13:49,756 3ecb3927 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/3ecb3927/pkb.log
2017-11-16 00:13:49,757 3ecb3927 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/3ecb3927/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user ajamato@ajamato2016.sea.corp.google.com
Not sending mail to unregistered user github@nakomis.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user valentyn@google.com
Not sending mail to unregistered user xumingmingv@gmail.com
ERROR: Unable to send to address: echauchot@gmail.com+jbonofre@apache.org
javax.mail.internet.AddressException: Domain contains illegal character in string ``echauchot@gmail.com+jbonofre@apache.org''
at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
at hudson.tasks.Mailer.stringToAddress(Mailer.java:206)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.toNormalizedAddress(MimeMessageBuilder.java:309)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:168)
at jenkins.plugins.mailer.tasks.MimeMessageBuilder.addRecipients(MimeMessageBuilder.java:160)
at hudson.tasks.MailSender.createEmptyMail(MailSender.java:389)
at hudson.tasks.MailSender.createFailureMail(MailSender.java:266)
at hudson.tasks.MailSender.createMail(MailSender.java:181)
at hudson.tasks.MailSender.run(MailSender.java:112)
at hudson.tasks.Mailer.perform(Mailer.java:170)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:736)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:682)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:627)
at hudson.model.Run.execute(Run.java:1762)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)