You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/01/12 12:18:14 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #781
See <https://builds.apache.org/job/beam_PerformanceTests_Python/781/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3405] Make maxNumRecords a long and validate if stream exists for
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a5b577930488a6b4b98695d5a4d579ebf023fa03 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a5b577930488a6b4b98695d5a4d579ebf023fa03
Commit message: "Merge pull request #4339 from iemejia/BEAM-3405-maxnumrecords"
> git rev-list 24c003496dc1930129169b29e5c677c6fb839dd5 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins725117301563998840.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7776940549419458737.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins59000305573569648.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5854346705211096711.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9074479739161292705.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-12 12:00:38,155 3aaf3122 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/3aaf3122/pkb.log
2018-01-12 12:00:38,155 3aaf3122 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-12 12:00:38,155 3aaf3122 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-12 12:00:38,569 3aaf3122 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-12 12:00:38,570 3aaf3122 MainThread INFO Initializing the edw service decoder
2018-01-12 12:00:38,691 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-12 12:00:38,694 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-12 12:00:38,695 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-12 12:00:44,370 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-12 12:07:46,868 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-12 12:07:46,881 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3aaf3122/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-12 12:17:46,885 3aaf3122 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3aaf3122/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-12 12:17:46,892 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3aaf3122/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/3aaf3122/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/3aaf3122/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/3aaf3122/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-12 12:17:46,892 3aaf3122 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-12 12:17:46,894 3aaf3122 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-12 12:17:46,895 3aaf3122 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-12 12:17:46,917 3aaf3122 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-12 12:17:46,918 3aaf3122 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/3aaf3122/pkb.log
2018-01-12 12:17:46,918 3aaf3122 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/3aaf3122/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #903
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/903/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 67234a15b56ddeed1d4dd90d276bad7e5003822c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 67234a15b56ddeed1d4dd90d276bad7e5003822c
Commit message: "Merge pull request #4603: [BEAM-3620] Remove older Kafka versions from build time support"
> git rev-list 67234a15b56ddeed1d4dd90d276bad7e5003822c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins604166585682635688.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8509965502448620521.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5308323065771750740.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5170901501968789861.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4359324185988578744.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5342159623770924607.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6155747632083455262.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3354289774328949299.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-12 00:00:28,136 f30965dc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f30965dc/pkb.log>
2018-02-12 00:00:28,140 f30965dc MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-12 00:00:28,140 f30965dc MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-12 00:00:28,387 f30965dc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-12 00:00:28,388 f30965dc MainThread INFO Initializing the edw service decoder
2018-02-12 00:00:28,515 f30965dc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-12 00:00:28,518 f30965dc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-12 00:00:28,519 f30965dc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-12 00:00:35,027 f30965dc MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-12 00:06:30,324 f30965dc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-12 00:06:30,340 f30965dc MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f30965dc/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-12 00:06:31,164 f30965dc MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f30965dc/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f30965dc/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-12 00:06:31,164 f30965dc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-12 00:06:31,165 f30965dc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-12 00:06:31,167 f30965dc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-12 00:06:31,192 f30965dc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-12 00:06:31,193 f30965dc MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f30965dc/pkb.log>
2018-02-12 00:06:31,193 f30965dc MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/f30965dc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #902
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/902/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 67234a15b56ddeed1d4dd90d276bad7e5003822c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 67234a15b56ddeed1d4dd90d276bad7e5003822c
Commit message: "Merge pull request #4603: [BEAM-3620] Remove older Kafka versions from build time support"
> git rev-list 67234a15b56ddeed1d4dd90d276bad7e5003822c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3757648903040482271.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3228057207488286290.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3213884175528723631.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins511432214497787349.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2745797791551187458.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1888428312129004922.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1228169121941007499.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7795122919481813182.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-11 18:00:43,170 ac7be54c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ac7be54c/pkb.log>
2018-02-11 18:00:43,171 ac7be54c MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-11 18:00:43,171 ac7be54c MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-11 18:00:43,452 ac7be54c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-11 18:00:43,453 ac7be54c MainThread INFO Initializing the edw service decoder
2018-02-11 18:00:43,580 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-11 18:00:43,583 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-11 18:00:43,583 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-11 18:00:50,957 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-11 18:06:52,892 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-11 18:06:52,907 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ac7be54c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-11 18:06:53,739 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ac7be54c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ac7be54c/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-11 18:06:53,739 ac7be54c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-11 18:06:53,740 ac7be54c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-11 18:06:53,742 ac7be54c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-11 18:06:53,794 ac7be54c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-11 18:06:53,795 ac7be54c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ac7be54c/pkb.log>
2018-02-11 18:06:53,795 ac7be54c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ac7be54c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #901
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/901/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 67234a15b56ddeed1d4dd90d276bad7e5003822c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 67234a15b56ddeed1d4dd90d276bad7e5003822c
Commit message: "Merge pull request #4603: [BEAM-3620] Remove older Kafka versions from build time support"
> git rev-list 67234a15b56ddeed1d4dd90d276bad7e5003822c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7169048929876364611.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1239047806826421178.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1342847975533484376.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6695522608736064042.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1771970811071226723.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4064506399078675033.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7620028332197685277.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6561961655189913424.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-11 12:00:37,540 3473cba0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3473cba0/pkb.log>
2018-02-11 12:00:37,540 3473cba0 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-11 12:00:37,541 3473cba0 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-11 12:00:37,966 3473cba0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-11 12:00:37,966 3473cba0 MainThread INFO Initializing the edw service decoder
2018-02-11 12:00:38,158 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-11 12:00:38,161 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-11 12:00:38,162 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-11 12:00:45,586 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-11 12:07:38,282 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-11 12:07:38,299 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3473cba0/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-11 12:07:39,333 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3473cba0/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3473cba0/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-11 12:07:39,334 3473cba0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-11 12:07:39,335 3473cba0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-11 12:07:39,337 3473cba0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-11 12:07:39,379 3473cba0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-11 12:07:39,379 3473cba0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3473cba0/pkb.log>
2018-02-11 12:07:39,380 3473cba0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3473cba0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #900
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/900/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 67234a15b56ddeed1d4dd90d276bad7e5003822c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 67234a15b56ddeed1d4dd90d276bad7e5003822c
Commit message: "Merge pull request #4603: [BEAM-3620] Remove older Kafka versions from build time support"
> git rev-list 67234a15b56ddeed1d4dd90d276bad7e5003822c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3471126445077613477.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins821368487411391225.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8865254936675380375.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2056938800619628828.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6350526133859850870.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5952185686393851707.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins410807587382128139.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1983527923294906269.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-11 06:00:39,306 ca5f7c10 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ca5f7c10/pkb.log>
2018-02-11 06:00:39,306 ca5f7c10 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-11 06:00:39,307 ca5f7c10 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-11 06:00:39,667 ca5f7c10 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-11 06:00:39,667 ca5f7c10 MainThread INFO Initializing the edw service decoder
2018-02-11 06:00:39,784 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-11 06:00:39,786 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-11 06:00:39,787 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-11 06:00:45,932 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-11 06:07:50,465 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-11 06:07:50,483 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ca5f7c10/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-11 06:07:51,393 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ca5f7c10/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ca5f7c10/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-11 06:07:51,393 ca5f7c10 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-11 06:07:51,394 ca5f7c10 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-11 06:07:51,396 ca5f7c10 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-11 06:07:51,410 ca5f7c10 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-11 06:07:51,410 ca5f7c10 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ca5f7c10/pkb.log>
2018-02-11 06:07:51,410 ca5f7c10 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ca5f7c10/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #899
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/899/display/redirect?page=changes>
Changes:
[rangadi] Remove older Kafka versions from build time support.
[rangadi] Move kafka-clients dependency to provided scope.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 67234a15b56ddeed1d4dd90d276bad7e5003822c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 67234a15b56ddeed1d4dd90d276bad7e5003822c
Commit message: "Merge pull request #4603: [BEAM-3620] Remove older Kafka versions from build time support"
> git rev-list 67ba5531c71d273dbbfa356f244bb2bb670b2bdf # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7671503057843117792.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5702978633318382297.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8768207372680132830.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3188304659742349448.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2208171215857969546.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6058804148053490386.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6355479506778333822.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5091178721560304762.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-11 00:00:30,661 69c8a6bb MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/69c8a6bb/pkb.log>
2018-02-11 00:00:30,662 69c8a6bb MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-11 00:00:30,662 69c8a6bb MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-11 00:00:30,859 69c8a6bb MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-11 00:00:30,859 69c8a6bb MainThread INFO Initializing the edw service decoder
2018-02-11 00:00:30,989 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-11 00:00:30,991 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-11 00:00:30,992 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-11 00:00:37,498 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-11 00:06:38,147 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-11 00:06:38,163 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/69c8a6bb/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-11 00:06:38,978 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/69c8a6bb/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/69c8a6bb/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-11 00:06:38,979 69c8a6bb MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-11 00:06:38,979 69c8a6bb MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-11 00:06:38,981 69c8a6bb MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-11 00:06:38,993 69c8a6bb MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-11 00:06:38,993 69c8a6bb MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/69c8a6bb/pkb.log>
2018-02-11 00:06:38,994 69c8a6bb MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/69c8a6bb/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #898
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/898/display/redirect?page=changes>
Changes:
[tgroh] Add a Primitive Impulse PTransform
[tgroh] Add a single-stage fusion implementation
[jb] [BEAM-3668] Quick workaround fix for netty conflict waiting better fix
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 67ba5531c71d273dbbfa356f244bb2bb670b2bdf (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 67ba5531c71d273dbbfa356f244bb2bb670b2bdf
Commit message: "Merge pull request #4653 from jbonofre/BEAM-3668"
> git rev-list 5056e06b7bbb6a70e00797e13de99a7492237be0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5837845295811352826.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins357812789014159322.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4962743173791763017.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6280206364873797403.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins309365571231561307.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5221231240337594637.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1184145187048581515.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6458270545645917834.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-10 18:00:55,782 6d6ebe5e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6d6ebe5e/pkb.log>
2018-02-10 18:00:55,782 6d6ebe5e MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-10 18:00:55,783 6d6ebe5e MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-10 18:00:56,127 6d6ebe5e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-10 18:00:56,128 6d6ebe5e MainThread INFO Initializing the edw service decoder
2018-02-10 18:00:56,262 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-10 18:00:56,265 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-10 18:00:56,266 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-10 18:01:03,525 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-10 18:08:02,246 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-10 18:08:02,263 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6d6ebe5e/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-10 18:08:03,280 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6d6ebe5e/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6d6ebe5e/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-10 18:08:03,281 6d6ebe5e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-10 18:08:03,282 6d6ebe5e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-10 18:08:03,284 6d6ebe5e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-10 18:08:03,340 6d6ebe5e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-10 18:08:03,341 6d6ebe5e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6d6ebe5e/pkb.log>
2018-02-10 18:08:03,341 6d6ebe5e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6d6ebe5e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #897
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/897/display/redirect?page=changes>
Changes:
[jb] [BEAM-793] Add backoff support in JdbcIO Write
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5056e06b7bbb6a70e00797e13de99a7492237be0 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5056e06b7bbb6a70e00797e13de99a7492237be0
Commit message: "Merge pull request #4500 from jbonofre/BEAM-793"
> git rev-list 959748d3f5851b2c55b76ba55168ef9fe3381518 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7880557541166470474.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1743051203744816413.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8762727440896774168.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3411991460026477517.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5942578897369914037.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1681661042754828352.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins587068756809056919.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1189020336645731964.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-10 12:00:41,249 2f530c7d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f530c7d/pkb.log>
2018-02-10 12:00:41,250 2f530c7d MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-10 12:00:41,251 2f530c7d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-10 12:00:41,605 2f530c7d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-10 12:00:41,605 2f530c7d MainThread INFO Initializing the edw service decoder
2018-02-10 12:00:41,744 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-10 12:00:41,747 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-10 12:00:41,747 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-10 12:00:48,786 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-10 12:08:00,829 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-10 12:08:00,846 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f530c7d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-10 12:08:01,983 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f530c7d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f530c7d/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-10 12:08:01,984 2f530c7d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-10 12:08:01,985 2f530c7d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-10 12:08:01,987 2f530c7d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-10 12:08:02,032 2f530c7d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-10 12:08:02,032 2f530c7d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f530c7d/pkb.log>
2018-02-10 12:08:02,033 2f530c7d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f530c7d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #896
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/896/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-419] Modifying FindBug comment.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 959748d3f5851b2c55b76ba55168ef9fe3381518 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 959748d3f5851b2c55b76ba55168ef9fe3381518
Commit message: "Merge pull request #3886: [BEAM-419] Fixing SE_BAD_FIELD FindBug in CombineFnUtil"
> git rev-list bd11416ef9d18b602fa725770cd14e50ade86b37 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7611235002658226397.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8311126126347061662.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5069431481101173821.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1156135078942191207.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1314426853739124805.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1587315293904364018.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1422584853050916181.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1289830231325606818.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-10 06:00:47,347 20ea4833 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/20ea4833/pkb.log>
2018-02-10 06:00:47,348 20ea4833 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-10 06:00:47,349 20ea4833 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-10 06:00:47,661 20ea4833 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-10 06:00:47,662 20ea4833 MainThread INFO Initializing the edw service decoder
2018-02-10 06:00:47,803 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-10 06:00:47,807 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-10 06:00:47,807 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-10 06:00:56,965 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-10 06:09:36,115 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-10 06:09:36,131 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/20ea4833/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-10 06:09:36,891 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/20ea4833/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/20ea4833/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-10 06:09:36,892 20ea4833 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-10 06:09:36,893 20ea4833 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-10 06:09:36,895 20ea4833 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-10 06:09:36,941 20ea4833 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-10 06:09:36,943 20ea4833 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/20ea4833/pkb.log>
2018-02-10 06:09:36,943 20ea4833 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/20ea4833/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #895
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/895/display/redirect?page=changes>
Changes:
[davidyan] Correct typo in SpannerIO.Write.withHost
[coheigea] Replace boolean ternary operator + simplify some Boolean expressions
[fjetumale] [BEAM-2817] BigQuery queries are allowed to run in either BATCH or
[ccy] Add switchable DirectRunner which uses the fast FnApiRunner when
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision bd11416ef9d18b602fa725770cd14e50ade86b37 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f bd11416ef9d18b602fa725770cd14e50ade86b37
Commit message: "Merge pull request #4606: [BEAM-2817] BigQuery queries are allowed to run in either BATCH or IN…"
> git rev-list 0feb813d571d4d0999ecd80a47c8dfa7879aab79 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5909182127762764510.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7599896080105841615.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins965182912956908602.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins342333022317973411.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5183718959860635422.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5083718059960511716.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8602961608231293055.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7856584117449084231.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-10 00:26:47,206 5e2693a8 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e2693a8/pkb.log>
2018-02-10 00:26:47,206 5e2693a8 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-10 00:26:47,207 5e2693a8 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-10 00:26:47,506 5e2693a8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-10 00:26:47,506 5e2693a8 MainThread INFO Initializing the edw service decoder
2018-02-10 00:26:47,634 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-10 00:26:47,637 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-10 00:26:47,637 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-10 00:26:54,000 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-10 00:33:46,704 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-10 00:33:46,717 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e2693a8/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-10 00:33:47,604 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e2693a8/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e2693a8/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-10 00:33:47,604 5e2693a8 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-10 00:33:47,605 5e2693a8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-10 00:33:47,607 5e2693a8 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-10 00:33:47,614 5e2693a8 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-10 00:33:47,614 5e2693a8 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e2693a8/pkb.log>
2018-02-10 00:33:47,614 5e2693a8 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5e2693a8/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user fjetumale@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #894
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/894/display/redirect?page=changes>
Changes:
[github] Consistantly show Python and pip versions in tests
[alan] [BEAM-3524] Automate testing using python sdk container built at head
[kenn] Increase gradle logging to --info
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0feb813d571d4d0999ecd80a47c8dfa7879aab79 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0feb813d571d4d0999ecd80a47c8dfa7879aab79
Commit message: "Merge pull request #4608 from cclauss/patch-1"
> git rev-list 8c5ebe4a4d158fa8c46640a0401376216a1d11c2 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5205755874128079228.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins672455186585422971.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6823201803618875764.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8581770376325453271.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5218788353438819255.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3961710072822870962.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins552121374309027749.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1932073505867760904.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-09 18:00:44,078 b261147f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b261147f/pkb.log>
2018-02-09 18:00:44,078 b261147f MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-09 18:00:44,079 b261147f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-09 18:00:44,512 b261147f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-09 18:00:44,512 b261147f MainThread INFO Initializing the edw service decoder
2018-02-09 18:00:44,633 b261147f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-09 18:00:44,636 b261147f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-09 18:00:44,636 b261147f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-09 18:00:51,688 b261147f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-09 18:10:03,128 b261147f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-09 18:10:03,151 b261147f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b261147f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-09 18:10:04,506 b261147f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b261147f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b261147f/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-09 18:10:04,507 b261147f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-09 18:10:04,508 b261147f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-09 18:10:04,511 b261147f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-09 18:10:04,527 b261147f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-09 18:10:04,527 b261147f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b261147f/pkb.log>
2018-02-09 18:10:04,528 b261147f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b261147f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #893
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/893/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8c5ebe4a4d158fa8c46640a0401376216a1d11c2 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8c5ebe4a4d158fa8c46640a0401376216a1d11c2
Commit message: "Merge pull request #4641 [BEAM-3644] Disable verbose typecoder warnings."
> git rev-list 8c5ebe4a4d158fa8c46640a0401376216a1d11c2 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4744648319913297523.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6681859014205225828.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8781162174100338612.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4806497145186551104.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4046458679286049698.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5655616452290914812.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5621263054228661377.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1854529810836080503.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-09 12:00:41,233 70ffe0bf MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/70ffe0bf/pkb.log>
2018-02-09 12:00:41,233 70ffe0bf MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-09 12:00:41,234 70ffe0bf MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-09 12:00:41,508 70ffe0bf MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-09 12:00:41,508 70ffe0bf MainThread INFO Initializing the edw service decoder
2018-02-09 12:00:41,638 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-09 12:00:41,641 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-09 12:00:41,641 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-09 12:00:48,213 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-09 12:07:50,332 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-09 12:07:50,348 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/70ffe0bf/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-09 12:07:51,368 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/70ffe0bf/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/70ffe0bf/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-09 12:07:51,369 70ffe0bf MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-09 12:07:51,370 70ffe0bf MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-09 12:07:51,372 70ffe0bf MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-09 12:07:51,425 70ffe0bf MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-09 12:07:51,426 70ffe0bf MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/70ffe0bf/pkb.log>
2018-02-09 12:07:51,426 70ffe0bf MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/70ffe0bf/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #892
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/892/display/redirect?page=changes>
Changes:
[robertwb] Disable verbose typecoder warnings.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8c5ebe4a4d158fa8c46640a0401376216a1d11c2 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8c5ebe4a4d158fa8c46640a0401376216a1d11c2
Commit message: "Merge pull request #4641 [BEAM-3644] Disable verbose typecoder warnings."
> git rev-list 3df248b48d3be78f8b37470d427921874f1647ea # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3994245533533100568.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5623854011421385852.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1276338473924140777.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6867253349889358569.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3228263551531746592.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4521068121228572727.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3527486950808499355.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4620683868762269801.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-09 06:00:47,105 57962014 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/57962014/pkb.log>
2018-02-09 06:00:47,106 57962014 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-09 06:00:47,107 57962014 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-09 06:00:47,455 57962014 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-09 06:00:47,456 57962014 MainThread INFO Initializing the edw service decoder
2018-02-09 06:00:47,666 57962014 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-09 06:00:47,670 57962014 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-09 06:00:47,671 57962014 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-09 06:00:54,665 57962014 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-09 06:09:39,017 57962014 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-09 06:09:39,042 57962014 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/57962014/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-09 06:09:40,409 57962014 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/57962014/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/57962014/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-09 06:09:40,410 57962014 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-09 06:09:40,411 57962014 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-09 06:09:40,413 57962014 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-09 06:09:40,466 57962014 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-09 06:09:40,466 57962014 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/57962014/pkb.log>
2018-02-09 06:09:40,466 57962014 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/57962014/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #891
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/891/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-3625] Enable DoFn params in Map, Filter, etc.
[apilloud] [BEAM-410] Sort PriorityQueue<QuantileBuffer> with explicit comparator
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3df248b48d3be78f8b37470d427921874f1647ea (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3df248b48d3be78f8b37470d427921874f1647ea
Commit message: "Merge pull request #4600: [BEAM-410] Sort PriorityQueue<QuantileBuffer> with explicit comparator"
> git rev-list 6792846069763770bbfd92a5eb61e64968522967 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2682626538948213751.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1769671578288419310.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1090966839980201130.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7847252738010885446.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5355353597151300352.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4927249454973804205.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3678146619602835959.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6174165307263512356.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-09 00:23:05,755 184b7ecc MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/184b7ecc/pkb.log>
2018-02-09 00:23:05,755 184b7ecc MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-09 00:23:05,756 184b7ecc MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-09 00:23:06,031 184b7ecc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-09 00:23:06,031 184b7ecc MainThread INFO Initializing the edw service decoder
2018-02-09 00:23:06,185 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-09 00:23:06,189 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-09 00:23:06,190 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-09 00:23:13,090 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-09 00:32:45,313 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-09 00:32:45,475 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/184b7ecc/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-09 00:32:46,568 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/184b7ecc/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/184b7ecc/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-09 00:32:46,568 184b7ecc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-09 00:32:46,569 184b7ecc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-09 00:32:46,570 184b7ecc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-09 00:32:46,583 184b7ecc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-09 00:32:46,584 184b7ecc MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/184b7ecc/pkb.log>
2018-02-09 00:32:46,584 184b7ecc MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/184b7ecc/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user apilloud@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #890
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/890/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6792846069763770bbfd92a5eb61e64968522967 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6792846069763770bbfd92a5eb61e64968522967
Commit message: "Merge pull request #4204: [BEAM-3157] Generate BeamRecord types from Pojos"
> git rev-list 6792846069763770bbfd92a5eb61e64968522967 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2163420454549967526.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins744456908022423087.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7321830917982415161.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8411628881629488443.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2766868182286632602.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3132807904252102636.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6108366762959005880.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7169202123431218349.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-08 18:00:35,577 8d6ee903 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8d6ee903/pkb.log>
2018-02-08 18:00:35,577 8d6ee903 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-08 18:00:35,578 8d6ee903 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-08 18:00:35,822 8d6ee903 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-08 18:00:35,822 8d6ee903 MainThread INFO Initializing the edw service decoder
2018-02-08 18:00:35,955 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-08 18:00:35,958 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-08 18:00:35,959 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-08 18:00:43,086 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-08 18:15:28,460 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-08 18:15:28,591 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8d6ee903/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-08 18:15:31,461 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8d6ee903/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8d6ee903/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-08 18:15:31,461 8d6ee903 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-08 18:15:31,500 8d6ee903 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-08 18:15:31,502 8d6ee903 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-08 18:15:31,541 8d6ee903 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-08 18:15:31,542 8d6ee903 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8d6ee903/pkb.log>
2018-02-08 18:15:31,542 8d6ee903 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/8d6ee903/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #889
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/889/display/redirect?page=changes>
Changes:
[kedin] [Schema Generation] Generate BeamRecordTypes based on pojos.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6792846069763770bbfd92a5eb61e64968522967 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6792846069763770bbfd92a5eb61e64968522967
Commit message: "Merge pull request #4204: [BEAM-3157] Generate BeamRecord types from Pojos"
> git rev-list f2c87bd385da2399938d6ff414176ea953c3c776 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4492582335843149834.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5170207305639875550.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3021090447035555221.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1102699554046592868.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3933460881392256366.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7140448655422899048.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3448737746940794476.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins897243359205820648.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-08 12:00:46,635 d5d63dba MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d5d63dba/pkb.log>
2018-02-08 12:00:46,636 d5d63dba MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-08 12:00:46,636 d5d63dba MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-08 12:00:46,872 d5d63dba MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-08 12:00:46,872 d5d63dba MainThread INFO Initializing the edw service decoder
2018-02-08 12:00:47,008 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-08 12:00:47,011 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-08 12:00:47,011 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-08 12:00:55,160 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-08 12:10:35,056 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-08 12:10:35,193 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d5d63dba/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-08 12:10:37,932 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d5d63dba/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d5d63dba/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-08 12:10:37,933 d5d63dba MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-08 12:10:37,934 d5d63dba MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-08 12:10:37,936 d5d63dba MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-08 12:10:37,960 d5d63dba MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-08 12:10:37,961 d5d63dba MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d5d63dba/pkb.log>
2018-02-08 12:10:37,961 d5d63dba MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d5d63dba/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #888
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/888/display/redirect?page=changes>
Changes:
[herohde] [BEAM-3457] Add Go Gradle precommit
[ccy] Update snippets to fix pickling and clarify encoding issues
[github] Revert "Update cloud spanner library to 0.29.0"
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2c87bd385da2399938d6ff414176ea953c3c776 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f2c87bd385da2399938d6ff414176ea953c3c776
Commit message: "Merge pull request #4635 from apache/revert-4139-depsupdate"
> git rev-list 6745a6119d609b4008e961912e84e932915bdb13 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5584927527848293471.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins265952456903428743.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3211178513825076765.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5811333024766613564.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4987289582997839078.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins975890875177527881.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins694638253452765003.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5644068225964698337.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-08 06:00:48,086 0f14d491 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0f14d491/pkb.log>
2018-02-08 06:00:48,086 0f14d491 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-08 06:00:48,087 0f14d491 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-08 06:00:48,417 0f14d491 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-08 06:00:48,417 0f14d491 MainThread INFO Initializing the edw service decoder
2018-02-08 06:00:48,554 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-08 06:00:48,556 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-08 06:00:48,557 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-08 06:00:57,463 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-08 06:08:58,823 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-08 06:08:58,913 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0f14d491/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-08 06:08:59,901 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0f14d491/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0f14d491/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-08 06:08:59,901 0f14d491 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-08 06:08:59,902 0f14d491 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-08 06:08:59,903 0f14d491 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-08 06:08:59,941 0f14d491 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-08 06:08:59,941 0f14d491 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0f14d491/pkb.log>
2018-02-08 06:08:59,941 0f14d491 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0f14d491/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #887
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/887/display/redirect?page=changes>
Changes:
[mairbek] Update cloud spanner library to 0.29.0
[mairbek] Fix test
[mairbek] More google-cloud-platform whitelisting
[mairbek] pom updates to make maven happy
[mairbek] Update netty deps
[mairbek] Fixed broken test
[ehudm] Add and migrate to HdfsCLI library for Python SDK.
[klk] Fix typo in gradle idea hints
[dkulp] [BEAM-3639] Update to gradle 4.5.1
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6745a6119d609b4008e961912e84e932915bdb13 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6745a6119d609b4008e961912e84e932915bdb13
Commit message: "[BEAM-3639] Update to gradle 4.5.1"
> git rev-list fab8ca0b1e07837d0a0d63d1957468c0401e01e7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1234899201070624103.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1860962156758772611.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6079312268737955276.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8141931892018767063.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4060032387073966504.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8282164355878853193.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9140247133996297060.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6632167746163433823.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-08 00:01:03,762 7575d28f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7575d28f/pkb.log>
2018-02-08 00:01:03,762 7575d28f MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-08 00:01:03,763 7575d28f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-08 00:01:04,106 7575d28f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-08 00:01:04,106 7575d28f MainThread INFO Initializing the edw service decoder
2018-02-08 00:01:04,317 7575d28f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-08 00:01:04,320 7575d28f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-08 00:01:04,320 7575d28f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-08 00:01:14,991 7575d28f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-08 00:11:01,386 7575d28f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-08 00:11:01,579 7575d28f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7575d28f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-08 00:11:05,381 7575d28f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7575d28f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (1).
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7575d28f/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395: RuntimeWarning: Unable to load plugin beam_test_plugin = test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-08 00:11:05,382 7575d28f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-08 00:11:05,433 7575d28f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-08 00:11:05,435 7575d28f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-08 00:11:05,449 7575d28f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-08 00:11:05,450 7575d28f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7575d28f/pkb.log>
2018-02-08 00:11:05,450 7575d28f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/7575d28f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user mairbek@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #886
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/886/display/redirect?page=changes>
Changes:
[jbonofre] Revert "Reinstate proto round trip in Java DirectRunner"
[tgroh] Update Assign Window URN Constant Name
[kedin] Rename BeamRecord -> Row, BeamRecordType -> RowType
[wcn] Modify BufferedElementCountingOutputStream to use a buffer pool for its
[klk] Fix stable name errors in HBaseIOTest
[aromanenko.dev] [BEAM-3291] Add Kinesis write transform
[coheigea] Remove unused private variables.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision fab8ca0b1e07837d0a0d63d1957468c0401e01e7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f fab8ca0b1e07837d0a0d63d1957468c0401e01e7
Commit message: "BEAM-3572: Modify BufferedElementCountingOutputStream to use a buffer pool"
> git rev-list 92a42020c9683db2fdc95a1c14023480d114b22c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3207206500187824607.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5514076754637105647.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8388522437644174647.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1024611615388094064.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8254005689068186400.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7465521318077960460.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1996540578611404362.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_AvroIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5435727245725883807.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-07 18:24:24,519 2005ba86 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/pkb.log>
2018-02-07 18:24:24,519 2005ba86 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-07 18:24:24,520 2005ba86 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-07 18:24:24,795 2005ba86 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-07 18:24:24,795 2005ba86 MainThread INFO Initializing the edw service decoder
2018-02-07 18:24:24,926 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-07 18:24:24,928 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-07 18:24:24,929 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-07 18:24:32,348 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-07 18:34:04,340 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-07 18:34:04,504 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-07 18:44:04,512 2005ba86 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-07 18:44:04,517 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-07 18:44:04,518 2005ba86 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-07 18:44:04,518 2005ba86 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-07 18:44:04,520 2005ba86 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-07 18:44:04,558 2005ba86 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-07 18:44:04,559 2005ba86 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/pkb.log>
2018-02-07 18:44:04,559 2005ba86 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2005ba86/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #885
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/885/display/redirect?page=changes>
Changes:
[iemejia] Add missing gradle build config for sdks/java/extensions/sketching
[iemejia] Fix type on shadowTest when it should be testShadow
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92a42020c9683db2fdc95a1c14023480d114b22c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92a42020c9683db2fdc95a1c14023480d114b22c
Commit message: "Merge pull request #4620: Add missing gradle build config for sdks/java/extensions/sketching"
> git rev-list d7bee8cf41dcc35f70135bd5baab25a6beaeaaca # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4982255140657798802.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins698083277274619708.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2770359466634960137.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8768629485695357982.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3746777345242774814.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins460400646874261257.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins188705779453671424.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_AvroIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2277852605938558318.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-07 12:00:40,573 cd6eef97 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/pkb.log>
2018-02-07 12:00:40,573 cd6eef97 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-07 12:00:40,574 cd6eef97 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-07 12:00:40,889 cd6eef97 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-07 12:00:40,889 cd6eef97 MainThread INFO Initializing the edw service decoder
2018-02-07 12:00:41,011 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-07 12:00:41,014 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-07 12:00:41,015 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-07 12:00:47,343 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-07 12:08:39,114 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-07 12:08:39,128 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-07 12:18:39,133 cd6eef97 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-07 12:18:39,141 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-07 12:18:39,142 cd6eef97 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-07 12:18:39,143 cd6eef97 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-07 12:18:39,146 cd6eef97 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-07 12:18:39,165 cd6eef97 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-07 12:18:39,166 cd6eef97 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/pkb.log>
2018-02-07 12:18:39,166 cd6eef97 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/cd6eef97/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #884
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/884/display/redirect?page=changes>
Changes:
[coheigea] BEAM-3624 - Remove collapsible if statements
[kenn] Switch FullWindowedValueCoder to bypass validation
[kedin] Refactor BeamRecordType and BeamRecord
[ccy] [BEAM-3635] Infer type hints on PTransformOverrides
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision d7bee8cf41dcc35f70135bd5baab25a6beaeaaca (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f d7bee8cf41dcc35f70135bd5baab25a6beaeaaca
Commit message: "BEAM-3624 - Remove collapsible if statements"
> git rev-list 92fd3545244d362f56f8f339d2630088f0d1af25 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6851501810467580239.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3234221321383151487.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3877608149176422395.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8836285189997510112.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1222711684998706117.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7797147748797238086.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1893639938720332123.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2176330282123289555.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-07 06:00:38,532 14dd814c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/pkb.log>
2018-02-07 06:00:38,532 14dd814c MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-07 06:00:38,533 14dd814c MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-07 06:00:38,783 14dd814c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-07 06:00:38,783 14dd814c MainThread INFO Initializing the edw service decoder
2018-02-07 06:00:38,907 14dd814c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-07 06:00:38,910 14dd814c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-07 06:00:38,910 14dd814c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-07 06:00:44,729 14dd814c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-07 06:07:47,444 14dd814c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-07 06:07:47,458 14dd814c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-07 06:17:47,463 14dd814c Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-07 06:17:47,469 14dd814c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-07 06:17:47,470 14dd814c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-07 06:17:47,471 14dd814c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-07 06:17:47,473 14dd814c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-07 06:17:47,531 14dd814c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-07 06:17:47,532 14dd814c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/pkb.log>
2018-02-07 06:17:47,532 14dd814c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/14dd814c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #883
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/883/display/redirect?page=changes>
Changes:
[rangadi] KafkaIO : move source and sink implemenations into own files.
[rangadi] minor
[kenn] Deprecate DoFnTester
[klk] Remove DoFnTester from core SDK tests
[kenn] Sickbay ApexRunner ParDoTranslatorTest.testAssertionFailure
[sidhom] Allow docker tag root to be specified as in Maven image build
[herohde] [BEAM-3457] Upgrade gogradle and fix thrift resolution issue
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 92fd3545244d362f56f8f339d2630088f0d1af25 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 92fd3545244d362f56f8f339d2630088f0d1af25
Commit message: "Merge pull request #4507: [BEAM-3159] Deprecated DoFnTester"
> git rev-list 282636275f3f91cc8a15aa08d8039bcb81d2f9d6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6690768810352157030.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2260185796931340341.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9060364513203332968.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1058027827327092644.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3853878879441015496.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5278826318546341380.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2446083152428459274.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3163884836472696358.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-07 00:00:44,408 d49f6e88 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/pkb.log>
2018-02-07 00:00:44,413 d49f6e88 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-07 00:00:44,414 d49f6e88 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-07 00:00:44,993 d49f6e88 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-07 00:00:44,993 d49f6e88 MainThread INFO Initializing the edw service decoder
2018-02-07 00:00:45,141 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-07 00:00:45,144 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-07 00:00:45,145 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-07 00:00:52,402 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-07 00:08:26,413 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-07 00:08:26,430 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-07 00:18:26,438 d49f6e88 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-07 00:18:26,448 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-07 00:18:26,448 d49f6e88 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-07 00:18:26,449 d49f6e88 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-07 00:18:26,452 d49f6e88 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-07 00:18:26,490 d49f6e88 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-07 00:18:26,491 d49f6e88 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/pkb.log>
2018-02-07 00:18:26,491 d49f6e88 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d49f6e88/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user herohde@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #882
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/882/display/redirect?page=changes>
Changes:
[Pablo] Renaming the ZERO element of DistributionResult to be IDENTITY_ELEMENT.
[ekirpichov] Adds PositionT and claim callback to RestrictionTracker
[ekirpichov] Changes OutputAndTimeBounded invoker to start checkpoint timer after
[ekirpichov] Compresses encoded GrowthState with Snappy - about 2x-3x more compact
[ekirpichov] InMemoryStateInternals.copy clones the values using the coder
[ekirpichov] Final fixups
[ekirpichov] Bump worker to 20180205
[klk] Sickbay flaky KinesisIO tests
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 282636275f3f91cc8a15aa08d8039bcb81d2f9d6 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 282636275f3f91cc8a15aa08d8039bcb81d2f9d6
Commit message: "Merge pull request #4483: [BEAM-3499, BEAM-2607] Gives the runner access to positions of SDF claimed blocks"
> git rev-list f3dba1af43b03e24419fb0f5b726479368959eef # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6897200577422977505.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2169439159314244710.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5999116873627816030.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins853925917496898066.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6830635159509522126.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8264345029084439122.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3112466105324635039.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8193146243904066857.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-06 18:37:33,920 76ce1c82 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/pkb.log>
2018-02-06 18:37:33,920 76ce1c82 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-06 18:37:33,921 76ce1c82 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-06 18:37:34,224 76ce1c82 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-06 18:37:34,225 76ce1c82 MainThread INFO Initializing the edw service decoder
2018-02-06 18:37:34,344 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-06 18:37:34,347 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-06 18:37:34,347 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-06 18:37:40,124 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-06 18:47:51,503 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-06 18:47:51,647 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-06 18:57:51,652 76ce1c82 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-06 18:57:51,660 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-06 18:57:51,660 76ce1c82 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-06 18:57:51,661 76ce1c82 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-06 18:57:51,663 76ce1c82 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-06 18:57:51,695 76ce1c82 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-06 18:57:51,696 76ce1c82 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/pkb.log>
2018-02-06 18:57:51,696 76ce1c82 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/76ce1c82/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #881
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/881/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3592] Fix spark-runner profile for Nexmark after move to Spark 2.x
[aljoscha.krettek] [BEAM-2806] Fix pipeline translation mode recognition in Flink Runner
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f3dba1af43b03e24419fb0f5b726479368959eef (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f3dba1af43b03e24419fb0f5b726479368959eef
Commit message: "Merge pull request #4558: Fixes streaming mode recognition in Flink Runner"
> git rev-list 8e2092ba28ad31517d65b3085840e6e9a46c83bf # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8283457316600383442.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3826027424712968978.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2824587446234089060.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6916924605340733553.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4315586847622913078.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3682595868735025943.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9098803196587715384.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins615784833758685593.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-06 12:00:39,170 a14d514d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/pkb.log>
2018-02-06 12:00:39,171 a14d514d MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-06 12:00:39,171 a14d514d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-06 12:00:39,432 a14d514d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-06 12:00:39,433 a14d514d MainThread INFO Initializing the edw service decoder
2018-02-06 12:00:39,557 a14d514d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-06 12:00:39,560 a14d514d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-06 12:00:39,561 a14d514d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-06 12:00:48,853 a14d514d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-06 12:10:29,374 a14d514d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-06 12:10:29,506 a14d514d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-06 12:20:29,510 a14d514d Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-06 12:20:29,542 a14d514d MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-06 12:20:29,543 a14d514d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-06 12:20:29,579 a14d514d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-06 12:20:29,598 a14d514d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-06 12:20:29,643 a14d514d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-06 12:20:29,643 a14d514d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/pkb.log>
2018-02-06 12:20:29,643 a14d514d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/a14d514d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #880
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/880/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8e2092ba28ad31517d65b3085840e6e9a46c83bf (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8e2092ba28ad31517d65b3085840e6e9a46c83bf
Commit message: "Merge pull request #4529 from [BEAM-3566] Replace apply_* hooks in DirectRunner with PTransformOverrides"
> git rev-list 8e2092ba28ad31517d65b3085840e6e9a46c83bf # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8050750355974562083.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4583841911990687607.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8261732458401309699.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2058054755165199310.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/aa/64/ea9f0f022bb9660b8a3c549754bf9105e05284ec408e93a65644522c1ecc/setuptools-38.5.0-py2.py3-none-any.whl#md5=f9582f22c550e37a0aa30a0dd2f72661
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins682193667744957802.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6282038092927760416.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8243584173378480296.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6777130014332859301.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-06 06:00:41,507 1af0b4b6 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/pkb.log>
2018-02-06 06:00:41,507 1af0b4b6 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-06 06:00:41,508 1af0b4b6 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-06 06:00:42,268 1af0b4b6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-06 06:00:42,268 1af0b4b6 MainThread INFO Initializing the edw service decoder
2018-02-06 06:00:42,418 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-06 06:00:42,422 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-06 06:00:42,422 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-06 06:00:49,311 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-06 06:09:02,638 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-06 06:09:02,691 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-06 06:19:02,704 1af0b4b6 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-06 06:19:02,715 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-06 06:19:02,716 1af0b4b6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-06 06:19:02,718 1af0b4b6 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-06 06:19:02,720 1af0b4b6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-06 06:19:02,734 1af0b4b6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-06 06:19:02,735 1af0b4b6 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/pkb.log>
2018-02-06 06:19:02,735 1af0b4b6 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1af0b4b6/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #879
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/879/display/redirect?page=changes>
Changes:
[Pablo] Moving User metrics to be in the PTransform proto for Fn API.
[ccy] [BEAM-3566] Replace apply_* hooks in DirectRunner with
[ccy] Address reviewer comments
[jiangkai] Covariance Functions
[robertwb] [BEAM-3183] Allow a callable as input to runner.run().
[sidhom] Fix gradle java sdk image build
[kenn] Add MoreFutures utility
[kenn] Switch runners/java-fn-execution from Guava futures to Java 8 futures
[kenn] Switch DataflowRunner from Guava futures to Java 8 futures
[kenn] Switch gcp-core from Guava futures to Java 8 futures
[kenn] Switch runners/core-construction-java from Guava futures to Java 8
[kenn] Switch AWS IO from Guava futures to Java 8 futures
[kenn] Switch BigTableIO from Guava futures to Java 8 futures
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8e2092ba28ad31517d65b3085840e6e9a46c83bf (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8e2092ba28ad31517d65b3085840e6e9a46c83bf
Commit message: "Merge pull request #4529 from [BEAM-3566] Replace apply_* hooks in DirectRunner with PTransformOverrides"
> git rev-list 2aca6ce2b69d76b9c1f8127d8647f9b1c7589219 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8695652388871773062.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8584536492190967477.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6593059882232593754.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1460524177545159586.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6518771498913675996.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8705279625347698268.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4529524435140134996.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8769023328582817437.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-06 00:21:59,058 e4d0b483 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/pkb.log>
2018-02-06 00:21:59,058 e4d0b483 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-06 00:21:59,059 e4d0b483 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-06 00:21:59,586 e4d0b483 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-06 00:21:59,587 e4d0b483 MainThread INFO Initializing the edw service decoder
2018-02-06 00:21:59,717 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-06 00:21:59,720 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-06 00:21:59,721 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-06 00:22:07,226 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-06 00:30:41,333 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-06 00:30:41,360 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-06 00:40:41,380 e4d0b483 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-06 00:40:41,393 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-06 00:40:41,394 e4d0b483 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-06 00:40:41,395 e4d0b483 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-06 00:40:41,398 e4d0b483 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-06 00:40:41,451 e4d0b483 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-06 00:40:41,452 e4d0b483 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/pkb.log>
2018-02-06 00:40:41,452 e4d0b483 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e4d0b483/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user jiangkai@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ccy@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #878
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/878/display/redirect?page=changes>
Changes:
[lcwik] Adjust gradle build dirs and hints to help IntelliJ (#4583)
[coheigea] BEAM-3618 - Remove extraneous "return" statement
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 2aca6ce2b69d76b9c1f8127d8647f9b1c7589219 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 2aca6ce2b69d76b9c1f8127d8647f9b1c7589219
Commit message: "BEAM-3618 - Remove extraneous "return" statement"
> git rev-list 93818abf901dd996aa30445457153811b13ece43 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9062689376651431394.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7049756771144757052.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7059146757956175519.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4007753776212593603.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins598886307047372180.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2497797300101069930.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6652894417205160442.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4419549884932646804.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-05 18:00:41,080 82a36762 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/pkb.log>
2018-02-05 18:00:41,081 82a36762 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-05 18:00:41,081 82a36762 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-05 18:00:41,486 82a36762 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-05 18:00:41,486 82a36762 MainThread INFO Initializing the edw service decoder
2018-02-05 18:00:41,604 82a36762 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-05 18:00:41,606 82a36762 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-05 18:00:41,607 82a36762 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-05 18:00:49,228 82a36762 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-05 18:10:07,139 82a36762 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-05 18:10:07,336 82a36762 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-05 18:20:07,341 82a36762 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-05 18:20:07,348 82a36762 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-05 18:20:07,349 82a36762 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-05 18:20:07,381 82a36762 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-05 18:20:07,384 82a36762 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-05 18:20:07,439 82a36762 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-05 18:20:07,440 82a36762 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/pkb.log>
2018-02-05 18:20:07,440 82a36762 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/82a36762/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #877
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/877/display/redirect?page=changes>
Changes:
[aljoscha.krettek] Remove erroneous cast in FlinkStreamingTransformTranslators
[aljoscha.krettek] [BEAM-3186] Correctly use deserialized timerService in Flink Runner
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 93818abf901dd996aa30445457153811b13ece43 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 93818abf901dd996aa30445457153811b13ece43
Commit message: "Merge pull request #4563: Fixes Flink timer service restore"
> git rev-list 24804e98d22180c1dc6603c8a437073ec2adde2d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins556338002970157487.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7139136910107254409.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1208891581456414687.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6510183300117871212.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/aa/64/ea9f0f022bb9660b8a3c549754bf9105e05284ec408e93a65644522c1ecc/setuptools-38.5.0-py2.py3-none-any.whl#md5=f9582f22c550e37a0aa30a0dd2f72661
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4816526190465977560.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2854794569014409044.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8086238549857582326.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins638013320271066560.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-05 12:00:39,822 2b705fc1 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/pkb.log>
2018-02-05 12:00:39,822 2b705fc1 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-05 12:00:39,823 2b705fc1 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-05 12:00:40,197 2b705fc1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-05 12:00:40,198 2b705fc1 MainThread INFO Initializing the edw service decoder
2018-02-05 12:00:40,338 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-05 12:00:40,341 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-05 12:00:40,341 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-05 12:00:46,822 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-05 12:08:37,516 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-05 12:08:37,680 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-05 12:18:37,689 2b705fc1 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-05 12:18:37,697 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-05 12:18:37,698 2b705fc1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-05 12:18:37,699 2b705fc1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-05 12:18:37,701 2b705fc1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-05 12:18:37,715 2b705fc1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-05 12:18:37,716 2b705fc1 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/pkb.log>
2018-02-05 12:18:37,716 2b705fc1 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2b705fc1/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user aljoscha.krettek@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #876
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/876/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 24804e98d22180c1dc6603c8a437073ec2adde2d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 24804e98d22180c1dc6603c8a437073ec2adde2d
Commit message: "Merge pull request #3978: [BEAM-230] Remove deprecated WindowedValue.valueInEmptyWindows"
> git rev-list 24804e98d22180c1dc6603c8a437073ec2adde2d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1000686184330290168.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1451447649313422212.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6293504148514462542.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5811312967770933527.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5392647905868717969.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4211587360238649336.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5910931739166069224.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2275696040518881075.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-05 06:00:57,365 e48b3389 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/pkb.log>
2018-02-05 06:00:57,366 e48b3389 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-05 06:00:57,366 e48b3389 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-05 06:00:57,729 e48b3389 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-05 06:00:57,730 e48b3389 MainThread INFO Initializing the edw service decoder
2018-02-05 06:00:57,863 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-05 06:00:57,866 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-05 06:00:57,867 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-05 06:01:04,359 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-05 06:07:09,776 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-05 06:07:09,792 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-05 06:17:09,798 e48b3389 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-05 06:17:09,804 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-05 06:17:09,805 e48b3389 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-05 06:17:09,806 e48b3389 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-05 06:17:09,808 e48b3389 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-05 06:17:09,863 e48b3389 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-05 06:17:09,864 e48b3389 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/pkb.log>
2018-02-05 06:17:09,864 e48b3389 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e48b3389/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #875
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/875/display/redirect?page=changes>
Changes:
[klk] google-java-format
[klk] Fix empty window assignments in Nexmark
[klk] Fix empty window assignment in FlattenEvaluatorFactoryTest
[klk] Switch DataflowRunner to its own private ValueInEmptyWindows
[klk] Remove deprecated valueInEmptyWindows
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 24804e98d22180c1dc6603c8a437073ec2adde2d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 24804e98d22180c1dc6603c8a437073ec2adde2d
Commit message: "Merge pull request #3978: [BEAM-230] Remove deprecated WindowedValue.valueInEmptyWindows"
> git rev-list 42ac62a067307173da061e7dbdbffd5e03672b96 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7357042400579950141.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8620550306112363940.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2376375369127758193.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins406750458963634177.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4216847549095899379.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins969792795530794222.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4821806783361639568.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1397246385080269035.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-05 00:00:30,835 1fc16c5c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/pkb.log>
2018-02-05 00:00:30,835 1fc16c5c MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-05 00:00:30,836 1fc16c5c MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-05 00:00:31,164 1fc16c5c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-05 00:00:31,165 1fc16c5c MainThread INFO Initializing the edw service decoder
2018-02-05 00:00:31,313 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-05 00:00:31,316 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-05 00:00:31,316 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-05 00:00:38,135 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-05 00:07:00,177 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-05 00:07:00,197 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-05 00:17:00,203 1fc16c5c Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-05 00:17:00,217 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-05 00:17:00,218 1fc16c5c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-05 00:17:00,220 1fc16c5c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-05 00:17:00,222 1fc16c5c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-05 00:17:00,268 1fc16c5c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-05 00:17:00,269 1fc16c5c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/pkb.log>
2018-02-05 00:17:00,269 1fc16c5c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/1fc16c5c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #874
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/874/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 42ac62a067307173da061e7dbdbffd5e03672b96 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 42ac62a067307173da061e7dbdbffd5e03672b96
Commit message: "Merge pull request #4310: [BEAM-3207] Create a standard location to enumerate and document URNs."
> git rev-list 42ac62a067307173da061e7dbdbffd5e03672b96 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1998642874727500945.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3494585915075410084.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1022198216222336467.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2273307006699146488.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3271030109051173147.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3149202802271322899.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5792401725580096929.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1885333415341802167.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-04 18:00:54,659 c2158cb2 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/pkb.log>
2018-02-04 18:00:54,659 c2158cb2 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-04 18:00:54,660 c2158cb2 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-04 18:00:55,057 c2158cb2 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-04 18:00:55,058 c2158cb2 MainThread INFO Initializing the edw service decoder
2018-02-04 18:00:55,188 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-04 18:00:55,191 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-04 18:00:55,191 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-04 18:01:02,049 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-04 18:07:12,582 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-04 18:07:12,597 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-04 18:17:12,603 c2158cb2 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-04 18:17:12,611 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-04 18:17:12,612 c2158cb2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-04 18:17:12,614 c2158cb2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-04 18:17:12,617 c2158cb2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-04 18:17:12,641 c2158cb2 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-04 18:17:12,642 c2158cb2 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/pkb.log>
2018-02-04 18:17:12,642 c2158cb2 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c2158cb2/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #873
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/873/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 42ac62a067307173da061e7dbdbffd5e03672b96 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 42ac62a067307173da061e7dbdbffd5e03672b96
Commit message: "Merge pull request #4310: [BEAM-3207] Create a standard location to enumerate and document URNs."
> git rev-list 42ac62a067307173da061e7dbdbffd5e03672b96 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5167828374419315688.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6692741726391341133.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5674308882467972790.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2466265682386546067.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5860113500095738154.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8907539543789420539.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4672702772560187214.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8837687549246930310.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-04 12:00:33,755 3f3718f0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/pkb.log>
2018-02-04 12:00:33,755 3f3718f0 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-04 12:00:33,755 3f3718f0 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-04 12:00:34,212 3f3718f0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-04 12:00:34,212 3f3718f0 MainThread INFO Initializing the edw service decoder
2018-02-04 12:00:34,334 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-04 12:00:34,337 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-04 12:00:34,338 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-04 12:00:41,082 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-04 12:07:34,815 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-04 12:07:34,832 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-04 12:17:34,837 3f3718f0 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-04 12:17:34,842 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-04 12:17:34,843 3f3718f0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-04 12:17:34,844 3f3718f0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-04 12:17:34,846 3f3718f0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-04 12:17:34,870 3f3718f0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-04 12:17:34,871 3f3718f0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/pkb.log>
2018-02-04 12:17:34,871 3f3718f0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/3f3718f0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #872
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/872/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 42ac62a067307173da061e7dbdbffd5e03672b96 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 42ac62a067307173da061e7dbdbffd5e03672b96
Commit message: "Merge pull request #4310: [BEAM-3207] Create a standard location to enumerate and document URNs."
> git rev-list 42ac62a067307173da061e7dbdbffd5e03672b96 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5880310312226461750.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins67790719460520932.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4446317790256788046.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7888241517095399516.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2441799989745222646.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5920163343609698122.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2863607965145908679.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7096112607678391855.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-04 06:00:46,035 144db064 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/pkb.log>
2018-02-04 06:00:46,035 144db064 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-04 06:00:46,036 144db064 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-04 06:00:46,388 144db064 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-04 06:00:46,388 144db064 MainThread INFO Initializing the edw service decoder
2018-02-04 06:00:46,587 144db064 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-04 06:00:46,590 144db064 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-04 06:00:46,590 144db064 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-04 06:00:55,641 144db064 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-04 06:10:29,772 144db064 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-04 06:10:29,796 144db064 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-04 06:20:29,803 144db064 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-04 06:20:29,809 144db064 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-04 06:20:29,810 144db064 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-04 06:20:29,811 144db064 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-04 06:20:29,814 144db064 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-04 06:20:29,837 144db064 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-04 06:20:29,838 144db064 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/pkb.log>
2018-02-04 06:20:29,838 144db064 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/144db064/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #871
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/871/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-3207] Create a standard location to enumerate and document URNs.
[robertwb] Revert URNs that are currently hard-coded in the Dataflow worker.
[kedin] [SQL] Add SqlTypeCoder, replace java.sql.Types
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 42ac62a067307173da061e7dbdbffd5e03672b96 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 42ac62a067307173da061e7dbdbffd5e03672b96
Commit message: "Merge pull request #4310: [BEAM-3207] Create a standard location to enumerate and document URNs."
> git rev-list 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6437454834343640197.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins41711574346754431.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5315939704187737314.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3831944567776251657.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/61/c8/3e04b8f555a598de6053bd6e07c6aa389e6801467599d166d17f984af6c0/setuptools-38.4.1-py2.py3-none-any.whl#md5=8f27eaab5b754ebf4fcf4d6daf78fe3e
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3340335583635539481.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6476966559704729581.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6618155839010429453.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4578139547402913986.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-04 00:00:33,075 c410b1ac MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/pkb.log>
2018-02-04 00:00:33,075 c410b1ac MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-04 00:00:33,076 c410b1ac MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-04 00:00:33,453 c410b1ac MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-04 00:00:33,453 c410b1ac MainThread INFO Initializing the edw service decoder
2018-02-04 00:00:33,594 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-04 00:00:33,597 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-04 00:00:33,597 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-04 00:00:40,159 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-04 00:08:01,228 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-04 00:08:01,256 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-04 00:18:01,266 c410b1ac Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-04 00:18:01,274 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-04 00:18:01,275 c410b1ac MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-04 00:18:01,276 c410b1ac MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-04 00:18:01,278 c410b1ac MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-04 00:18:01,300 c410b1ac MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-04 00:18:01,301 c410b1ac MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/pkb.log>
2018-02-04 00:18:01,302 c410b1ac MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/c410b1ac/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #870
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/870/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6
Commit message: "Merge pull request #4578: [BEAM-3603] Adds a ReadAll transform to tfrecordio"
> git rev-list 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8771076743768557403.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3946484048348957082.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2901512981262046497.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins943096339721720063.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/61/c8/3e04b8f555a598de6053bd6e07c6aa389e6801467599d166d17f984af6c0/setuptools-38.4.1-py2.py3-none-any.whl#md5=8f27eaab5b754ebf4fcf4d6daf78fe3e
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6974599252394770215.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3727627421156537624.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1928236069391221347.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5737512816177221973.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-03 18:01:11,824 5d2d62eb MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/pkb.log>
2018-02-03 18:01:11,825 5d2d62eb MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-03 18:01:11,825 5d2d62eb MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-03 18:01:12,548 5d2d62eb MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-03 18:01:12,549 5d2d62eb MainThread INFO Initializing the edw service decoder
2018-02-03 18:01:12,682 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-03 18:01:12,686 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-03 18:01:12,686 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-03 18:01:19,238 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-03 18:10:11,110 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-03 18:10:11,131 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-03 18:20:11,140 5d2d62eb Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-03 18:20:11,152 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-03 18:20:11,153 5d2d62eb MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-03 18:20:11,154 5d2d62eb MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-03 18:20:11,157 5d2d62eb MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-03 18:20:11,179 5d2d62eb MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-03 18:20:11,181 5d2d62eb MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/pkb.log>
2018-02-03 18:20:11,181 5d2d62eb MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5d2d62eb/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #869
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/869/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6
Commit message: "Merge pull request #4578: [BEAM-3603] Adds a ReadAll transform to tfrecordio"
> git rev-list 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2073589549577809106.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2786882898192093351.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8340508999685444342.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7902913812140214287.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6467281769186736362.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3543063612501090115.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8077532679361228947.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2755693545058241009.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-03 12:00:52,906 b4034a72 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/pkb.log>
2018-02-03 12:00:52,907 b4034a72 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-03 12:00:52,907 b4034a72 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-03 12:00:53,114 b4034a72 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-03 12:00:53,115 b4034a72 MainThread INFO Initializing the edw service decoder
2018-02-03 12:00:53,244 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-03 12:00:53,247 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-03 12:00:53,248 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-03 12:00:59,716 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-03 12:07:00,280 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-03 12:07:00,295 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-03 12:17:00,301 b4034a72 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-03 12:17:00,308 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-03 12:17:00,309 b4034a72 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-03 12:17:00,310 b4034a72 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-03 12:17:00,312 b4034a72 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-03 12:17:00,378 b4034a72 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-03 12:17:00,378 b4034a72 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/pkb.log>
2018-02-03 12:17:00,379 b4034a72 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b4034a72/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #868
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/868/display/redirect?page=changes>
Changes:
[Pablo] Logging deviation from sampling expectation. This will allow to track
[Pablo] Adding a static getter for RuntimeValueProvider.
[Pablo] Addressing comments.
[github] Removing unnecessary code.
[cclauss] global INT64_MAX, INT64_MIN to placate linters
[tgroh] Add a LocalArtifactStagingLocation
[tgroh] Add LocalArtifactStagingLocation#forExisting
[tgroh] Add an ArtifactRetrievalService interface
[tgroh] Implement a Local ArtifactRetrievalService
[chamikara] Adds a ReadAll transform to tfrecordio.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8130c20dec5cfbda120ad0b6b8025526fa5ed6d6
Commit message: "Merge pull request #4578: [BEAM-3603] Adds a ReadAll transform to tfrecordio"
> git rev-list 5e6520a841d3b1b2959e21969f551a8b66d298dc # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins628077633323940676.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4710412204411799586.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4064543524272982369.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9182006881955816846.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4062785761493375369.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3425052077116360519.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5243976828921554068.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5248578548941422045.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-03 06:13:43,443 0875dd9c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/pkb.log>
2018-02-03 06:13:43,443 0875dd9c MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-03 06:13:43,444 0875dd9c MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-03 06:13:43,956 0875dd9c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-03 06:13:43,957 0875dd9c MainThread INFO Initializing the edw service decoder
2018-02-03 06:13:44,084 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-03 06:13:44,088 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-03 06:13:44,088 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-03 06:13:50,934 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-03 06:20:24,247 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-03 06:20:24,268 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-03 06:30:24,275 0875dd9c Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-03 06:30:24,282 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-03 06:30:24,282 0875dd9c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-03 06:30:24,283 0875dd9c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-03 06:30:24,286 0875dd9c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-03 06:30:24,317 0875dd9c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-03 06:30:24,318 0875dd9c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/pkb.log>
2018-02-03 06:30:24,318 0875dd9c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0875dd9c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user cclauss@bluewin.ch
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #867
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/867/display/redirect?page=changes>
Changes:
[tgroh] Add SdkHarnessClientControlService
[tgroh] Update Synchronization in FnApiControlClient
[coheigea] BEAM-3593 - Remove methods that just call super()
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5e6520a841d3b1b2959e21969f551a8b66d298dc (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 5e6520a841d3b1b2959e21969f551a8b66d298dc
Commit message: "Merge pull request #4569: Add `SdkHarnessClientControlService`"
> git rev-list 21cc003914272883daff36a352bb6a57fcc6dfe2 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4995573222077604108.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4040151245508403802.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4179322237074404973.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1797352361547077022.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1788630527240470200.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5109004949940480837.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3636629700814087455.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4863392481424538317.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-03 00:01:29,340 e109731f MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/pkb.log>
2018-02-03 00:01:29,341 e109731f MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-03 00:01:29,341 e109731f MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-03 00:01:29,649 e109731f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-03 00:01:29,649 e109731f MainThread INFO Initializing the edw service decoder
2018-02-03 00:01:29,803 e109731f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-03 00:01:29,806 e109731f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-03 00:01:29,807 e109731f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-03 00:01:36,797 e109731f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-03 00:08:40,752 e109731f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-03 00:08:40,771 e109731f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-03 00:18:40,778 e109731f Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-03 00:18:40,785 e109731f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-03 00:18:40,786 e109731f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-03 00:18:40,787 e109731f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-03 00:18:40,791 e109731f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-03 00:18:40,842 e109731f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-03 00:18:40,843 e109731f MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/pkb.log>
2018-02-03 00:18:40,843 e109731f MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e109731f/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #866
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/866/display/redirect?page=changes>
Changes:
[robertwb] Reduce the flakiness of the state sampler progress metrics.
[kedin] [SQL] Inherit windowing strategy from the input in Aggregate operation
[jbonofre] [BEAM-3551] Define compiler -parameters flag in the default options
[lcwik] Move off of deprecated method in Guava.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 21cc003914272883daff36a352bb6a57fcc6dfe2 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 21cc003914272883daff36a352bb6a57fcc6dfe2
Commit message: "Merge pull request #4546: [SQL] Inherit windowing strategy from the input in Aggregate operation"
> git rev-list 9cf86bcebcdbd8d5a84777cf2871597f0ba1b951 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2578158365985461499.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4574589399925147037.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5728173047620967361.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1561706708763531326.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins905477893136226474.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins593717921323410652.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2305585672070199438.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5805481535065959264.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-02 20:23:08,892 5749e819 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/pkb.log>
2018-02-02 20:23:08,892 5749e819 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-02 20:23:08,893 5749e819 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-02 20:23:09,190 5749e819 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-02 20:23:09,190 5749e819 MainThread INFO Initializing the edw service decoder
2018-02-02 20:23:09,317 5749e819 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-02 20:23:09,320 5749e819 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-02 20:23:09,321 5749e819 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-02 20:23:16,075 5749e819 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-02 20:30:16,830 5749e819 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-02 20:30:16,846 5749e819 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-02 20:40:16,852 5749e819 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-02 20:40:16,858 5749e819 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-02 20:40:16,859 5749e819 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-02 20:40:16,861 5749e819 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-02 20:40:16,863 5749e819 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-02 20:40:16,918 5749e819 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-02 20:40:16,919 5749e819 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/pkb.log>
2018-02-02 20:40:16,919 5749e819 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/5749e819/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #865
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/865/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 9cf86bcebcdbd8d5a84777cf2871597f0ba1b951 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9cf86bcebcdbd8d5a84777cf2871597f0ba1b951
Commit message: "Merge pull request #4301: Introduces the Wait transform"
> git rev-list 9cf86bcebcdbd8d5a84777cf2871597f0ba1b951 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5356997287317944221.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6477910650308969277.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4785595407424347375.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5022164939614466206.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8896889881970279591.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8072121964405520941.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2755885300766538609.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4808045664766274731.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-02 12:00:44,666 0571c09d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/pkb.log>
2018-02-02 12:00:44,666 0571c09d MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-02 12:00:44,666 0571c09d MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-02 12:00:44,947 0571c09d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-02 12:00:44,948 0571c09d MainThread INFO Initializing the edw service decoder
2018-02-02 12:00:45,089 0571c09d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-02 12:00:45,092 0571c09d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-02 12:00:45,093 0571c09d MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-02 12:00:51,686 0571c09d MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-02 12:08:01,821 0571c09d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-02 12:08:01,841 0571c09d MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-02 12:18:01,847 0571c09d Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-02 12:18:01,854 0571c09d MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-02 12:18:01,854 0571c09d MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-02 12:18:01,856 0571c09d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-02 12:18:01,858 0571c09d MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-02 12:18:01,883 0571c09d MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-02 12:18:01,883 0571c09d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/pkb.log>
2018-02-02 12:18:01,884 0571c09d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0571c09d/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #864
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/864/display/redirect?page=changes>
Changes:
[ekirpichov] Introduces the Wait transform
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 9cf86bcebcdbd8d5a84777cf2871597f0ba1b951 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 9cf86bcebcdbd8d5a84777cf2871597f0ba1b951
Commit message: "Merge pull request #4301: Introduces the Wait transform"
> git rev-list e34fee13a12ad1bc8694c7f8d1bdedc9afb88e17 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8125196431367118470.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8894134476788846354.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3038674210411406039.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3256991282615249445.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/75/d1/5abca4ccf61a7ab86c255dd315fb96e566fbf9b5d3a480e72c93e8ec2802/setuptools-38.4.0-py2.py3-none-any.whl#md5=a5c6620a59f19f2d5d32bdca18c7b47e
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1656602075560595163.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2718874454610923649.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7035198647196741168.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_AvroIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4793311173795423573.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-02 06:00:36,972 509949ba MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/pkb.log>
2018-02-02 06:00:36,972 509949ba MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-02 06:00:36,973 509949ba MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-02 06:00:37,279 509949ba MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-02 06:00:37,280 509949ba MainThread INFO Initializing the edw service decoder
2018-02-02 06:00:37,442 509949ba MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-02 06:00:37,445 509949ba MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-02 06:00:37,445 509949ba MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-02 06:00:45,112 509949ba MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-02 06:07:56,953 509949ba MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-02 06:07:56,967 509949ba MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-02 06:17:56,972 509949ba Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-02 06:17:56,980 509949ba MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-02 06:17:56,981 509949ba MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-02 06:17:56,982 509949ba MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-02 06:17:56,984 509949ba MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-02 06:17:57,029 509949ba MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-02 06:17:57,029 509949ba MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/pkb.log>
2018-02-02 06:17:57,029 509949ba MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/509949ba/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #863
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/863/display/redirect?page=changes>
Changes:
[ehudm] Split out buffered read and write code from gcsio.
[github] import logging for line 1163
[dkulp] [BEAM-3562] Update to Checkstyle 8.7
[klk] Encourage a good description in a good spot on a PR description.
[tgroh] Add QueryablePipeline
[gene] Changing FileNaming to public to allow for usage in lambdas/inheritance
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e34fee13a12ad1bc8694c7f8d1bdedc9afb88e17 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e34fee13a12ad1bc8694c7f8d1bdedc9afb88e17
Commit message: "Merge pull request #4471: [BEAM-3099] Split out BufferedReader and BufferedWriter from gcsio."
> git rev-list 51da92cce831755866e5c017802532929fdfd872 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5273086330297496473.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9159302210169260110.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4553115145016364535.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8831732620183427392.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/75/d1/5abca4ccf61a7ab86c255dd315fb96e566fbf9b5d3a480e72c93e8ec2802/setuptools-38.4.0-py2.py3-none-any.whl#md5=a5c6620a59f19f2d5d32bdca18c7b47e
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins510103526591437663.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins403324489383911251.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5057966014719650190.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_AvroIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins615137555199172700.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-02 01:37:18,166 ccded0f2 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/pkb.log>
2018-02-02 01:37:18,166 ccded0f2 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-02 01:37:18,167 ccded0f2 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-02 01:37:18,554 ccded0f2 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-02 01:37:18,555 ccded0f2 MainThread INFO Initializing the edw service decoder
2018-02-02 01:37:18,738 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-02 01:37:18,742 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-02 01:37:18,743 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-02 01:37:29,271 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-02 01:47:49,592 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-02 01:47:49,682 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-02 01:57:49,689 ccded0f2 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-02 01:57:49,698 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:166: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-02 01:57:49,698 ccded0f2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-02 01:57:49,750 ccded0f2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-02 01:57:49,762 ccded0f2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-02 01:57:49,798 ccded0f2 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-02 01:57:49,799 ccded0f2 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/pkb.log>
2018-02-02 01:57:49,799 ccded0f2 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/ccded0f2/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user gene@telligent-data.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #862
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/862/display/redirect?page=changes>
Changes:
[github] Fix undefined names: exc_info --> self.exc_info
[lcwik] Change info to debug statement
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 51da92cce831755866e5c017802532929fdfd872 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 51da92cce831755866e5c017802532929fdfd872
Commit message: "Merge pull request #4559 from cclauss/patch-1"
> git rev-list 82e5e944ba143c23acf377bfd7a850046be68ea7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8399523810663267945.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6785706637888403514.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6583095736507094117.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1548845153372327499.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/75/d1/5abca4ccf61a7ab86c255dd315fb96e566fbf9b5d3a480e72c93e8ec2802/setuptools-38.4.0-py2.py3-none-any.whl#md5=a5c6620a59f19f2d5d32bdca18c7b47e
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins17278353873698400.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4052711741288962951.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5183352582043366966.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5870062200948541622.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-01 18:17:31,307 6cf75b44 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/pkb.log>
2018-02-01 18:17:31,308 6cf75b44 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-01 18:17:31,309 6cf75b44 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-01 18:17:31,631 6cf75b44 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-01 18:17:31,632 6cf75b44 MainThread INFO Initializing the edw service decoder
2018-02-01 18:17:31,785 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-01 18:17:31,789 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-01 18:17:31,790 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-01 18:17:39,567 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-01 18:27:03,802 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-01 18:27:03,866 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-01 18:37:03,875 6cf75b44 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-01 18:37:03,881 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-01 18:37:03,882 6cf75b44 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-01 18:37:03,883 6cf75b44 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-01 18:37:03,885 6cf75b44 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-01 18:37:03,930 6cf75b44 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-01 18:37:03,930 6cf75b44 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/pkb.log>
2018-02-01 18:37:03,930 6cf75b44 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/6cf75b44/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #861
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/861/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 82e5e944ba143c23acf377bfd7a850046be68ea7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 82e5e944ba143c23acf377bfd7a850046be68ea7
Commit message: "[BEAM-3249] Do not assume build directory is within build/, use the project defined build dir."
> git rev-list 82e5e944ba143c23acf377bfd7a850046be68ea7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7621189792462985900.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3528553918196377218.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6919199800811630340.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5374407727404169519.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from https://pypi.python.org/packages/75/d1/5abca4ccf61a7ab86c255dd315fb96e566fbf9b5d3a480e72c93e8ec2802/setuptools-38.4.0-py2.py3-none-any.whl#md5=a5c6620a59f19f2d5d32bdca18c7b47e
Downloading/unpacking pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4244646903443425979.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7228499570969448497.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8356437278448163096.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_AvroIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8287135835227915017.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-01 12:01:28,979 839a94a7 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/pkb.log>
2018-02-01 12:01:28,980 839a94a7 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-01 12:01:28,981 839a94a7 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-01 12:01:29,390 839a94a7 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-01 12:01:29,390 839a94a7 MainThread INFO Initializing the edw service decoder
2018-02-01 12:01:29,574 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-01 12:01:29,579 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-01 12:01:29,580 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-01 12:01:37,560 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-01 12:12:09,142 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-01 12:12:09,217 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-01 12:22:09,223 839a94a7 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-01 12:22:09,230 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-01 12:22:09,230 839a94a7 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-01 12:22:09,365 839a94a7 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-01 12:22:09,385 839a94a7 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-01 12:22:09,391 839a94a7 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-01 12:22:09,392 839a94a7 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/pkb.log>
2018-02-01 12:22:09,392 839a94a7 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/839a94a7/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user joey.baruch@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #860
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/860/display/redirect?page=changes>
Changes:
[mariagh] Support argparse-style choices for ValueProvider
[kenn] Move TestCountingSource to appropriate location
[kenn] Sickbay ApexRunner gradle WordCountIT
[lcwik] [BEAM-3249] Make sure that all java projects package tests. Also package
[lcwik] [BEAM-3249] Do not assume build directory is within build/, use the
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 82e5e944ba143c23acf377bfd7a850046be68ea7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 82e5e944ba143c23acf377bfd7a850046be68ea7
Commit message: "[BEAM-3249] Do not assume build directory is within build/, use the project defined build dir."
> git rev-list dd9a5137cf141a5e800f5507960dc5b4a9409928 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5123726834537282199.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4098707992788243420.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5147422277530907854.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4619134471988841339.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4750011823615010929.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9119307115754719554.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.12.1
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2423309418091910117.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3273734690530919444.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-01 06:00:41,323 213a6ee9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/pkb.log>
2018-02-01 06:00:41,323 213a6ee9 MainThread INFO PerfKitBenchmarker version: v1.12.0-260-ge324e61
2018-02-01 06:00:41,324 213a6ee9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-01 06:00:41,698 213a6ee9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-02-01 06:00:41,699 213a6ee9 MainThread INFO Initializing the edw service decoder
2018-02-01 06:00:41,838 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-01 06:00:41,841 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-02-01 06:00:41,842 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-02-01 06:00:48,433 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-01 06:07:37,962 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-02-01 06:07:37,979 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-01 06:17:37,985 213a6ee9 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-02-01 06:17:37,992 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-02-01 06:17:37,993 213a6ee9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-01 06:17:37,995 213a6ee9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-02-01 06:17:37,999 213a6ee9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-02-01 06:17:38,068 213a6ee9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-01 06:17:38,069 213a6ee9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/pkb.log>
2018-02-01 06:17:38,070 213a6ee9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/213a6ee9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user rober@frantil.com
Build failed in Jenkins: beam_PerformanceTests_Python #859
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/859/display/redirect?page=changes>
Changes:
[wcn] Update generated version of Fn API code.
[robertwb] [BEAM-3490] Make runtime type checking code runner agnostic.
[github] get the query from configuration not options
[dariusz.aniszewski] use build $WORKSPACE as pkb temp_dir and update pip and setuptools in
[kenn] google-java-format
[kenn] Fix Distinct null pointer error with speculative triggers
[robertwb] Direct runner fixes.
[lcwik] [BEAM-2926] Add support for side inputs to the runner harness.
[kenn] Sickbay flakey KinesisReaderTest
------------------------------------------
[...truncated 115.05 KB...]
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/dtype_transfer.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/mapping.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/flagsobject.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/nditer_constr.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/convert_datatype.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/array_assign_scalar.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/conversion_utils.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/common.c
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_templ.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/ucsnarrow.c
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/multiarray/einsum.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/number.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/numpyos.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/datetime.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/datetime_strings.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/ctors.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/temp_elide.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/nditer_api.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/iterators.c
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/multiarray/scalartypes.c
x86_64-linux-gnu-gcc: numpy/core/src/private/mem_overlap.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/array_assign.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/nditer_pywrap.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/datetime_busday.c
x86_64-linux-gnu-gcc: numpy/core/src/multiarray/alloc.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/alloc.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/arrayobject.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/arraytypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign_scalar.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign_array.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/buffer.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/calculation.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/compiled_base.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/common.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/convert.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/convert_datatype.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/conversion_utils.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/ctors.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_strings.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_busday.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_busdaycal.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/descriptor.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/dtype_transfer.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/einsum.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/flagsobject.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/getset.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/hashdescr.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/item_selection.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/iterators.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/lowlevel_strided_loops.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/mapping.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_templ.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_api.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_constr.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_pywrap.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/number.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/numpyos.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/refcount.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/sequence.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/shape.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/scalarapi.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/scalartypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/temp_elide.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/usertypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/ucsnarrow.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/vdot.o build/temp.linux-x86_64-2.7/numpy/core/src/private/mem_overlap.o build/temp.linux-x86_64-2.7/numpy/core/src/private/ufunc_override.o -Lbuild/temp.linux-x86_64-2.7 -lnpymath -lnpysort -lm -o build/lib.linux-x86_64-2.7/numpy/core/multiarray.so
building 'numpy.core.umath' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/umath -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: numpy/core/src/umath/ufunc_type_resolution.c
x86_64-linux-gnu-gcc: numpy/core/src/umath/ufunc_object.c
numpy/core/src/umath/ufunc_object.c: In function ‘PyUFunc_GenericReduction’:
numpy/core/src/umath/ufunc_object.c:3912:15: warning: unused variable ‘out_obj’ [-Wunused-variable]
PyObject *out_obj = NULL;
^
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/umath/scalarmath.c
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/umath/loops.c
x86_64-linux-gnu-gcc: numpy/core/src/umath/umathmodule.c
x86_64-linux-gnu-gcc: numpy/core/src/umath/reduction.c
x86_64-linux-gnu-gcc: numpy/core/src/private/mem_overlap.c
x86_64-linux-gnu-gcc: numpy/core/src/private/ufunc_override.c
x86_64-linux-gnu-gcc: numpy/core/src/umath/override.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/numpy/core/src/umath/umathmodule.o build/temp.linux-x86_64-2.7/numpy/core/src/umath/reduction.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/umath/loops.o build/temp.linux-x86_64-2.7/numpy/core/src/umath/ufunc_object.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/umath/scalarmath.o build/temp.linux-x86_64-2.7/numpy/core/src/umath/ufunc_type_resolution.o build/temp.linux-x86_64-2.7/numpy/core/src/umath/override.o build/temp.linux-x86_64-2.7/numpy/core/src/private/mem_overlap.o build/temp.linux-x86_64-2.7/numpy/core/src/private/ufunc_override.o -Lbuild/temp.linux-x86_64-2.7 -lnpymath -lm -o build/lib.linux-x86_64-2.7/numpy/core/umath.so
building 'numpy.core.umath_tests' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/umath/umath_tests.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/umath/umath_tests.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/core/umath_tests.so
building 'numpy.core.test_rational' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/umath/test_rational.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/umath/test_rational.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/core/test_rational.so
building 'numpy.core.struct_ufunc_test' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/umath/struct_ufunc_test.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/umath/struct_ufunc_test.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/core/struct_ufunc_test.so
building 'numpy.core.multiarray_tests' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: numpy/core/src/private/mem_overlap.c
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/multiarray/multiarray_tests.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/multiarray_tests.o build/temp.linux-x86_64-2.7/numpy/core/src/private/mem_overlap.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/core/multiarray_tests.so
building 'numpy.core.operand_flag_tests' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/core/src/umath/operand_flag_tests.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/umath/operand_flag_tests.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/core/operand_flag_tests.so
building 'numpy.fft.fftpack_lite' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: numpy/fft/fftpack.c
x86_64-linux-gnu-gcc: numpy/fft/fftpack_litemodule.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/numpy/fft/fftpack_litemodule.o build/temp.linux-x86_64-2.7/numpy/fft/fftpack.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/fft/fftpack_lite.so
building 'numpy.linalg.lapack_lite' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/python_xerbla.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_blas.c
numpy/linalg/lapack_lite/f2c_blas.c: In function ‘cgemm_’:
numpy/linalg/lapack_lite/f2c_blas.c:389:20: warning: variable ‘ncola’ set but not used [-Wunused-but-set-variable]
static integer ncola;
^
numpy/linalg/lapack_lite/f2c_blas.c: In function ‘dgemm_’:
numpy/linalg/lapack_lite/f2c_blas.c:6895:20: warning: variable ‘ncola’ set but not used [-Wunused-but-set-variable]
static integer ncola;
^
numpy/linalg/lapack_lite/f2c_blas.c: In function ‘sgemm_’:
numpy/linalg/lapack_lite/f2c_blas.c:11514:20: warning: variable ‘ncola’ set but not used [-Wunused-but-set-variable]
static integer ncola;
^
numpy/linalg/lapack_lite/f2c_blas.c: In function ‘zgemm_’:
numpy/linalg/lapack_lite/f2c_blas.c:15736:20: warning: variable ‘ncola’ set but not used [-Wunused-but-set-variable]
static integer ncola;
^
x86_64-linux-gnu-gcc: numpy/linalg/lapack_litemodule.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_z_lapack.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_config.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_d_lapack.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_s_lapack.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_lapack.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c_c_lapack.c
x86_64-linux-gnu-gcc: numpy/linalg/lapack_lite/f2c.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/numpy/linalg/lapack_litemodule.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/python_xerbla.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_z_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_c_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_d_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_s_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_blas.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_config.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/linalg/lapack_lite.so
building 'numpy.linalg._umath_linalg' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: build/src.linux-x86_64-2.7/numpy/linalg/umath_linalg.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/linalg/umath_linalg.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/python_xerbla.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_z_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_c_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_d_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_s_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_lapack.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_blas.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c_config.o build/temp.linux-x86_64-2.7/numpy/linalg/lapack_lite/f2c.o -Lbuild/temp.linux-x86_64-2.7 -lnpymath -o build/lib.linux-x86_64-2.7/numpy/linalg/_umath_linalg.so
building 'numpy.random.mtrand' extension
compiling C sources
C compiler: x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC
compile options: '-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python2.7 -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-2.7/numpy/core/src/private -Ibuild/src.linux-x86_64-2.7/numpy/core/src/npymath -c'
x86_64-linux-gnu-gcc: numpy/random/mtrand/distributions.c
x86_64-linux-gnu-gcc: numpy/random/mtrand/mtrand.c
x86_64-linux-gnu-gcc: numpy/random/mtrand/initarray.c
x86_64-linux-gnu-gcc: numpy/random/mtrand/randomkit.c
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/numpy/random/mtrand/mtrand.o build/temp.linux-x86_64-2.7/numpy/random/mtrand/randomkit.o build/temp.linux-x86_64-2.7/numpy/random/mtrand/initarray.o build/temp.linux-x86_64-2.7/numpy/random/mtrand/distributions.o -Lbuild/temp.linux-x86_64-2.7 -o build/lib.linux-x86_64-2.7/numpy/random/mtrand.so
Creating build/scripts.linux-x86_64-2.7/f2py
adding 'build/scripts.linux-x86_64-2.7/f2py' to scripts
changing mode of build/scripts.linux-x86_64-2.7/f2py from 644 to 755
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
changing mode of <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/f2py> to 755
Successfully installed numpy
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5932658324104086406.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.4.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.4.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2327021453264058199.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/740db536'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user wcn@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user rober@frantil.com
Build failed in Jenkins: beam_PerformanceTests_Python #858
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/858/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3578] SQL module build breaks because of missing dependency
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7b7e8dde29f26f0a1a77b6a894af07e97599d24e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7b7e8dde29f26f0a1a77b6a894af07e97599d24e
Commit message: "Merge pull request #4549 from iemejia/BEAM-2578-sql-dep"
> git rev-list 645e1b51e1c83ec4b0b54a774d6615624f3c42ed # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5646063345368849976.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4421141886392967595.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5963219026004771230.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7994379640720859627.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1601815199770930547.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Downloading/unpacking absl-py (from -r PerfKitBenchmarker/requirements.txt (line 14))
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/build/absl-py/setup.py)> egg_info for package absl-py
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Downloading/unpacking colorlog[windows]==2.6.0 (from -r PerfKitBenchmarker/requirements.txt (line 17))
Downloading colorlog-2.6.0-py2.py3-none-any.whl
Downloading/unpacking blinker>=1.3 (from -r PerfKitBenchmarker/requirements.txt (line 18))
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/build/blinker/setup.py)> egg_info for package blinker
Downloading/unpacking futures>=3.0.3 (from -r PerfKitBenchmarker/requirements.txt (line 19))
Downloading futures-3.2.0-py2-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Downloading/unpacking pint>=0.7 (from -r PerfKitBenchmarker/requirements.txt (line 21))
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/build/pint/setup.py)> egg_info for package pint
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'docs/_themes/.git'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
warning: no previously-included files matching '*__pycache__*' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '.travis-exclude.yml' found anywhere in distribution
Downloading/unpacking numpy (from -r PerfKitBenchmarker/requirements.txt (line 22))
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/build/numpy/setup.py)> egg_info for package numpy
Running from numpy source directory.
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
warning: no previously-included files matching '*.swp' found anywhere in distribution
warning: no previously-included files matching '*.bak' found anywhere in distribution
warning: no previously-included files matching '*~' found anywhere in distribution
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Downloading/unpacking contextlib2>=0.5.1 (from -r PerfKitBenchmarker/requirements.txt (line 24))
Downloading contextlib2-0.5.5-py2.py3-none-any.whl
Downloading/unpacking pywinrm (from -r PerfKitBenchmarker/requirements.txt (line 25))
Downloading pywinrm-0.3.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): colorama in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied (use --upgrade to upgrade): requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Downloading/unpacking requests-ntlm>=0.3.0 (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Downloading requests_ntlm-1.1.0-py2.py3-none-any.whl
Downloading/unpacking xmltodict (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Downloading xmltodict-0.11.0-py2.py3-none-any.whl
Downloading/unpacking cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/build/cryptography/setup.py)> egg_info for package cryptography
error in cryptography setup command: Invalid environment marker: python_version < '3'
Complete output from command python setup.py egg_info:
error in cryptography setup command: Invalid environment marker: python_version < '3'
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/build/cryptography>
Storing debug log for failure in /home/jenkins/.pip/pip.log
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user rober@frantil.com
Build failed in Jenkins: beam_PerformanceTests_Python #857
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/857/display/redirect>
------------------------------------------
[...truncated 722.49 KB...]
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ beam-sdks-java-extensions-sql ---
[INFO] Compiling 90 source files to /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/test-classes
[INFO] /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlTimestampMinusTimestampExpressionTest.java: Some input files use or override a deprecated API.
[INFO] /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlTimestampMinusTimestampExpressionTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/TestUtils.java: Some input files use unchecked or unsafe operations.
[INFO] /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/TestUtils.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ beam-sdks-java-extensions-sql ---
[INFO]
[INFO] --- maven-surefire-plugin:2.20.1:test (default-test) @ beam-sdks-java-extensions-sql ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:regex-properties (render-artifact-id) @ beam-sdks-java-extensions-sql ---
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:report (report) @ beam-sdks-java-extensions-sql ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ beam-sdks-java-extensions-sql ---
[INFO] Building jar: /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/beam-sdks-java-extensions-sql-2.4.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ beam-sdks-java-extensions-sql ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (default-test-jar) @ beam-sdks-java-extensions-sql ---
[INFO] Building jar: /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/beam-sdks-java-extensions-sql-2.4.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-shade-plugin:3.1.0:shade (bundle-and-repackage) @ beam-sdks-java-extensions-sql ---
[INFO] Including org.apache.calcite:calcite-core:jar:1.13.0 in the shaded jar.
[INFO] Excluding commons-dbcp:commons-dbcp:jar:1.4 from the shaded jar.
[INFO] Excluding commons-pool:commons-pool:jar:1.5.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.hydromatic:aggdesigner-algorithm:jar:6.0 from the shaded jar.
[INFO] Excluding commons-lang:commons-lang:jar:2.4 from the shaded jar.
[INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded jar.
[INFO] Including org.codehaus.janino:janino:jar:2.7.6 in the shaded jar.
[INFO] Including org.codehaus.janino:commons-compiler:jar:2.7.6 in the shaded jar.
[INFO] Including org.apache.calcite:calcite-linq4j:jar:1.13.0 in the shaded jar.
[INFO] Including org.apache.calcite.avatica:avatica-core:jar:1.10.0 in the shaded jar.
[INFO] Including org.apache.calcite.avatica:avatica-metrics:jar:1.10.0 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:3.2.0 in the shaded jar.
[INFO] Excluding org.apache.httpcomponents:httpclient:jar:4.5.2 from the shaded jar.
[INFO] Excluding commons-codec:commons-codec:jar:1.9 from the shaded jar.
[INFO] Excluding org.apache.httpcomponents:httpcore:jar:4.4.4 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-csv:jar:1.4 from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-extensions-join-library:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.alibaba:fastjson:jar:1.2.12 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-runners-direct-java:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding org.apache.beam:beam-model-pipeline:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding org.apache.beam:beam-runners-local-java-core:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding org.apache.beam:beam-runners-core-construction-java:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding org.apache.beam:beam-model-job-management:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-protobuf-lite:jar:1.2.0 from the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java-util:jar:3.2.0 in the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-runners-core-java:jar:2.4.0-SNAPSHOT from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/beam-sdks-java-extensions-sql-2.4.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/beam-sdks-java-extensions-sql-2.4.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/beam-sdks-java-extensions-sql-2.4.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/target/beam-sdks-java-extensions-sql-2.4.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/java/extensions/sql/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:analyze-only (default) @ beam-sdks-java-extensions-sql ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 11.977 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 6.063 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.094 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 9.912 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.097 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 4.962 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.188 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 9.296 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.121 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 31.721 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.840 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.079 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 4.141 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.084 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.722 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 5.196 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 10.281 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 4.487 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.079 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Amazon Web Services SUCCESS [ 3.668 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.969 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 12.562 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.907 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.359 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 3.351 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.705 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.741 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.636 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 2.951 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 2.246 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.756 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.704 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 9.179 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.101 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 5.522 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 3.905 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.485 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Input Format SUCCESS [ 13.662 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.859 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 9.406 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.558 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 3.025 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 3.021 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.642 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.926 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.372 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 5.388 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 4.829 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.839 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.631 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.045 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 10.304 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.063 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.904 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 28.895 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.512 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.253 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.918 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.934 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... FAILURE [ 17.371 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SKIPPED
[INFO] Apache Beam :: SDKs :: Python ...................... SKIPPED
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SKIPPED
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ SKIPPED
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:16 min
[INFO] Finished at: 2018-01-31T12:05:59Z
[INFO] Final Memory: 331M/1855M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.2:analyze-only (default) on project beam-sdks-java-extensions-sql: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-sdks-java-extensions-sql
STDERR: log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
2018-01-31 12:06:00,809 799ac185 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-31 12:06:00,818 799ac185 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 175, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 265, in _BuildPythonCommand
'apache-beam*.tar.gz')
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 290, in _FindFiles
raise RuntimeError('No such directory: %s' % base_path)
RuntimeError: No such directory: /tmp/perfkitbenchmarker/runs/799ac185/beam/sdks/python/target
2018-01-31 12:06:00,820 799ac185 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-31 12:06:00,822 799ac185 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-31 12:06:00,864 799ac185 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-31 12:06:00,865 799ac185 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/799ac185/pkb.log
2018-01-31 12:06:00,865 799ac185 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/799ac185/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user rober@frantil.com
Build failed in Jenkins: beam_PerformanceTests_Python #856
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/856/display/redirect?page=changes>
Changes:
[kedin] [SQL] Refactor Variance
[kedin] [Nexmark][SQL] Implement sql query 3
[tgroh] Add CoderTranslatorRegistrar
[chamikara] Updates PTransform overriding to create a new AppliedPTransform object
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 645e1b51e1c83ec4b0b54a774d6615624f3c42ed (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 645e1b51e1c83ec4b0b54a774d6615624f3c42ed
Commit message: "Merge pull request #4448: [SQL] Refactor Variance"
> git rev-list 34d8bc9079a5a4b3d3a5476b2d4b48072be0dff4 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5499283578059472838.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins527850684334946878.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1097187561501222493.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7306039795591851249.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2310814785634687574.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3740887293761302288.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.4.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.4.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8026123637253369796.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/a3c5fcb2'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user rober@frantil.com
Build failed in Jenkins: beam_PerformanceTests_Python #855
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/855/display/redirect?page=changes>
Changes:
[tgroh] Add slf4j_simple to the top level Gradle build
[tgroh] Implement FnService in FnApiControlClientPoolService
[tgroh] Add a Timeout to GrpcDataService#send
[tgroh] Use a Data Service in SdkHarnessClient
[XuMingmin] [BEAM-3525] Fix KafkaIO metric (#4524)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 34d8bc9079a5a4b3d3a5476b2d4b48072be0dff4 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 34d8bc9079a5a4b3d3a5476b2d4b48072be0dff4
Commit message: "[BEAM-3525] Fix KafkaIO metric (#4524)"
> git rev-list a7226cba18fb0366bc8b1d516ccab1fa65d753ed # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3405383127224712091.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1526764464314917463.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6993315847457157443.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins152595654224580139.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8022485230757275373.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2377601178439960802.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5179054166457745200.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6492448938520059062.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-31 00:21:43,742 e45a6cf9 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/pkb.log>
2018-01-31 00:21:43,742 e45a6cf9 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-31 00:21:43,743 e45a6cf9 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-31 00:21:44,024 e45a6cf9 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-31 00:21:44,024 e45a6cf9 MainThread INFO Initializing the edw service decoder
2018-01-31 00:21:44,136 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-31 00:21:44,139 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-31 00:21:44,139 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-31 00:21:49,515 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-31 00:29:34,842 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-31 00:29:34,973 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-01-31 00:39:34,978 e45a6cf9 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".>
2018-01-31 00:39:34,983 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz.> Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/beam/sdks/python/apache_beam/io/gcp/gcsio.py>:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/beam/sdks/python/apache_beam/coders/typecoders.py>:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-31 00:39:34,983 e45a6cf9 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-31 00:39:34,984 e45a6cf9 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-31 00:39:34,986 e45a6cf9 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-31 00:39:35,016 e45a6cf9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-31 00:39:35,016 e45a6cf9 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/pkb.log>
2018-01-31 00:39:35,017 e45a6cf9 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/e45a6cf9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Not sending mail to unregistered user rober@frantil.com
Build failed in Jenkins: beam_PerformanceTests_Python #854
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/854/display/redirect?page=changes>
Changes:
[kenn] Add a test for an event time timer loop in ParDo
[pawel.pk.kaczmarczyk] [BEAM-2469] Handling Kinesis shards splits and merges
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a7226cba18fb0366bc8b1d516ccab1fa65d753ed (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a7226cba18fb0366bc8b1d516ccab1fa65d753ed
Commit message: "Merge pull request #3745: [BEAM-2791] Add a test for an event time timer loop in ParDo"
> git rev-list dc24beb9f3651dc05749b8fac9bc6f6acacf7022 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8862466731767327857.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3706425129200079782.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2344431064361173270.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8852298751330289034.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1557445552538784123.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6838029990535680905.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.4.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.4.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7188446938789655402.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/22b053d0'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user pawel.pk.kaczmarczyk@gmail.com
Build failed in Jenkins: beam_PerformanceTests_Python #853
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/853/display/redirect?page=changes>
Changes:
[joey.baruch] Add javadoc to ConsoleIO
[lcwik] [BEAM-2500, BEAM-3249] Add amazon-web-services gradle build rules.
[github] [BEAM-3557] Sets parent pointer of AppliedPTransform objects correctly
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision dc24beb9f3651dc05749b8fac9bc6f6acacf7022 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f dc24beb9f3651dc05749b8fac9bc6f6acacf7022
Commit message: "Merge pull request #3839: Add javadoc to ConsoleIO"
> git rev-list 21623d892302717a3ad5b3dbc447ebc38268bb29 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8710428979684075735.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6186107474566433790.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4473822791037069439.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6458451769936506786.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins702031711613407868.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5776093088808095569.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3587796979900524964.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/ba55e730'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user joey.baruch@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #852
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/852/display/redirect?page=changes>
Changes:
[Pablo] Tracking of time spent reading side inputs, and bytes read in Dataflow.
[Pablo] Fix lint issues
[Pablo] Fixing counter names
[Pablo] Using comments to improve changes
[Pablo] Placing feature behind an experiment flag.
[Pablo] Fixing lint issue
[Pablo] Rebasing
[Pablo] Added cythonization and test
[Pablo] Fixing lint issue
[Pablo] Modifying the experiment flag check to reduce performance impact.
[Pablo] Addressing comments. Profile pending.
[Pablo] Reducing impact - Tracking only Bytes if experiment is active.
[Pablo] Addressing comments
[Pablo] Improved IO Target documentation
[Pablo] Improving documentation
[Pablo] Addressing comments.
[iemejia] Update maven-shade-plugin version to 3.1.0
[iemejia] Update maven-compiler-plugin version to 3.7.0
[iemejia] Update maven-dependency-plugin version to 3.0.2
[iemejia] Update maven-surefire-plugin version to 2.20.1
[iemejia] Update maven-failsafe-plugin version to 2.20.1
[iemejia] Update maven-assembly-plugin version to 3.1.0
[iemejia] Update versions-maven-plugin version to 2.5
[iemejia] Update findbugs-maven-plugin version to 3.0.5
[iemejia] Update license-maven-plugin version to 1.14
[iemejia] Update jacoco-maven-plugin version to 0.7.9
[iemejia] Update dockerfile-maven-plugin version to 1.3.6
[iemejia] Add maven-enforcer-plugin version to pluginManagement
[iemejia] Fix warning on using directly parent.version on Nexmark
[iemejia] Remove warnings on repeated maven-jar-plugin invocation / deps
[iemejia] Remove warning on defining <prerequisites> for non-maven plugin project
[iemejia] Update parent POM to version 19
[iemejia] Remove repeated dependency in hadoop-input-format module
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 21623d892302717a3ad5b3dbc447ebc38268bb29 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 21623d892302717a3ad5b3dbc447ebc38268bb29
Commit message: "Merge pull request #4248 from pabloem/sideinput-io-tracking"
> git rev-list 4593d9762296f3a012a1e2781d9b9c29e16ffb75 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3147268004546537466.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7570788211995001094.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2280517727401798932.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8047374131527490269.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4924399819211355605.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins50570113106854499.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.4.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3102666789253397711.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/68307518'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #851
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/851/display/redirect?page=changes>
Changes:
[jbonofre] [maven-release-plugin] prepare branch release-2.3.0
[jbonofre] [maven-release-plugin] prepare for next development iteration
[jbonofre] Update Python SDK version post release
[iemejia] Fix dependency order error, harness must be built before container
[robertwb] [BEAM-3537] Allow more general eager in-process pipeline execution
[lcwik] [BEAM-3550] Add --awsServiceEndpoint option and use with S3 filesystem.
[lcwik] Fixes for sdks/java/core for the eclipse compiler
[lcwik] Add some m2e lifecycle things to get the various maven plugins running
[lcwik] Set the proper package for the Snippets example so eclipse won't raise
[lcwik] Add some template args and direct casts to help the eclipse compiler
[lcwik] Incorporate reviews and rebase on latest master
[lcwik] Return no environment more often
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 4593d9762296f3a012a1e2781d9b9c29e16ffb75 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 4593d9762296f3a012a1e2781d9b9c29e16ffb75
Commit message: "Return no environment more often"
> git rev-list 7c6b9940b8bf38d4f268461668a534e8073f24d1 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4696691776500400784.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8372936504170253866.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3791383555985721874.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5244726140192741698.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2915832529797528346.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1187062362212265856.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.4.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5419806167338051224.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-30 00:05:02,946 f2cfc2ed MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f2cfc2ed/pkb.log
2018-01-30 00:05:02,946 f2cfc2ed MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-30 00:05:02,947 f2cfc2ed MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-30 00:05:03,288 f2cfc2ed MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-30 00:05:03,289 f2cfc2ed MainThread INFO Initializing the edw service decoder
2018-01-30 00:05:03,452 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-30 00:05:03,455 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-30 00:05:03,456 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-30 00:05:11,157 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-30 00:13:11,038 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-30 00:13:11,053 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f2cfc2ed/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz
2018-01-30 00:23:11,061 f2cfc2ed Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f2cfc2ed/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz".
2018-01-30 00:23:11,068 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f2cfc2ed/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f2cfc2ed/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/f2cfc2ed/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f2cfc2ed/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-30 00:23:11,069 f2cfc2ed MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-30 00:23:11,070 f2cfc2ed MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-30 00:23:11,072 f2cfc2ed MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-30 00:23:11,124 f2cfc2ed MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-30 00:23:11,125 f2cfc2ed MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f2cfc2ed/pkb.log
2018-01-30 00:23:11,126 f2cfc2ed MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f2cfc2ed/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #850
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/850/display/redirect?page=changes>
Changes:
[coheigea] BEAM-3560 - Switch to use BigInteger/BigDecimal.ZERO/ONE/TEN
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7c6b9940b8bf38d4f268461668a534e8073f24d1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7c6b9940b8bf38d4f268461668a534e8073f24d1
Commit message: "Merge pull request #4521 from coheigea/bigint"
> git rev-list c72c3ccd48524cfc79425bd4e501e92727a9d33f # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9034931664950049118.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2944179343111176698.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8245770526278588329.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3931887902394291965.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6023486401391103753.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4922690404873475329.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1720462600719062083.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/394361a3'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #849
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/849/display/redirect?page=changes>
Changes:
[aromanenko.dev] [BEAM-3539] BigtableIO.Write javadoc of some methods is incorrect
[jbonofre] [BEAM-3551] Add -parameters flag to the compiler
[jbonofre] [BEAM-3551] Align gradle java compile task options with the
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision c72c3ccd48524cfc79425bd4e501e92727a9d33f (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f c72c3ccd48524cfc79425bd4e501e92727a9d33f
Commit message: "Merge pull request #4516: [BEAM-3551] Add -parameters flag to the compiler"
> git rev-list c891a3c4923501698c671f547387d766d8bbceb3 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2536918568436137773.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5658253738846514915.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3226063886041451458.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3243932012956698903.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6274176961114699382.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2849647288392855587.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins468605621506472282.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-29 12:00:31,051 db43ef14 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/db43ef14/pkb.log
2018-01-29 12:00:31,059 db43ef14 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-29 12:00:31,060 db43ef14 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-29 12:00:31,779 db43ef14 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-29 12:00:31,779 db43ef14 MainThread INFO Initializing the edw service decoder
2018-01-29 12:00:31,934 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-29 12:00:31,938 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-29 12:00:31,939 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-29 12:00:38,643 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-29 12:08:38,967 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-29 12:08:38,986 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/db43ef14/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-29 12:18:38,994 db43ef14 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/db43ef14/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-29 12:18:39,003 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/db43ef14/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/db43ef14/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/db43ef14/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/db43ef14/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-29 12:18:39,004 db43ef14 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-29 12:18:39,005 db43ef14 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-29 12:18:39,007 db43ef14 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-29 12:18:39,045 db43ef14 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-29 12:18:39,045 db43ef14 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/db43ef14/pkb.log
2018-01-29 12:18:39,045 db43ef14 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/db43ef14/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user aromanenko.dev@gmail.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #848
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/848/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision c891a3c4923501698c671f547387d766d8bbceb3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f c891a3c4923501698c671f547387d766d8bbceb3
Commit message: "Merge pull request #4513 from mottyg1/master"
> git rev-list c891a3c4923501698c671f547387d766d8bbceb3 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8580181271172778459.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1548349920272893547.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5992364454596232429.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins961867889216644898.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5870833588534096551.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4189937546311931051.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2953234107980341237.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/6a2d710f'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #847
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/847/display/redirect?page=changes>
Changes:
[Mottyg1] [BEAM-675] Introduce message mapper in JmsIO
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision c891a3c4923501698c671f547387d766d8bbceb3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f c891a3c4923501698c671f547387d766d8bbceb3
Commit message: "Merge pull request #4513 from mottyg1/master"
> git rev-list f553121a80aae0bd1a686bef769540d10bbf12ee # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1454959664231766965.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9021682239272819802.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4169463823618931369.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1292491824215593516.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2463579196775984363.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5323935911494525098.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins649150449134330239.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-29 00:00:35,190 b0ba6c1f MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b0ba6c1f/pkb.log
2018-01-29 00:00:35,190 b0ba6c1f MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-29 00:00:35,192 b0ba6c1f MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-29 00:00:35,970 b0ba6c1f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-29 00:00:35,971 b0ba6c1f MainThread INFO Initializing the edw service decoder
2018-01-29 00:00:36,121 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-29 00:00:36,124 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-29 00:00:36,125 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-29 00:00:43,229 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-29 00:08:20,675 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-29 00:08:20,692 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b0ba6c1f/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-29 00:18:20,700 b0ba6c1f Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b0ba6c1f/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-29 00:18:20,709 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b0ba6c1f/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/b0ba6c1f/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/b0ba6c1f/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/b0ba6c1f/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-29 00:18:20,710 b0ba6c1f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-29 00:18:20,711 b0ba6c1f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-29 00:18:20,714 b0ba6c1f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-29 00:18:20,744 b0ba6c1f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-29 00:18:20,746 b0ba6c1f MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b0ba6c1f/pkb.log
2018-01-29 00:18:20,746 b0ba6c1f MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b0ba6c1f/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user Mottyg1@gmail.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #846
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/846/display/redirect?page=changes>
Changes:
[rangadi] Remove an unused test dependency in KafkaIO.
[jbonofre] Typo fix in MinimalWordCount example
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f553121a80aae0bd1a686bef769540d10bbf12ee (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f553121a80aae0bd1a686bef769540d10bbf12ee
Commit message: "Merge pull request #4505 from rangadi/dependency"
> git rev-list e59c19d2e3f4684f1ceabbfc816c2e4b3b54a6a2 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5334729771891139129.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6411411903418882507.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins429449959775380731.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2479825453106020306.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8173569133376213339.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4307829172889245678.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5137598823695254602.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/4f8adc2e'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #845
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/845/display/redirect?page=changes>
Changes:
[jbonofre] [BEAM-2271] Exclude files not intended to be in the releases source
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e59c19d2e3f4684f1ceabbfc816c2e4b3b54a6a2 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e59c19d2e3f4684f1ceabbfc816c2e4b3b54a6a2
Commit message: "Merge pull request #4498 from jbonofre/BEAM-2271"
> git rev-list ba32a8cb68d6fefd41ea318f2a951e3335d8751f # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3007893279034378597.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5399655416843519833.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7900103010337135893.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7007696102135055607.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3165882296944134314.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins463930299044012310.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4385505132933681161.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/c4ffc306'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #844
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/844/display/redirect?page=changes>
Changes:
[iemejia] Fix missing gearpump module activated only on Java 8 profile
[iemejia] Add missing amazon-web-services module from javadoc
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ba32a8cb68d6fefd41ea318f2a951e3335d8751f (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ba32a8cb68d6fefd41ea318f2a951e3335d8751f
Commit message: "Merge pull request #4512 from iemejia/fix-java-modules"
> git rev-list 29859eb54d05b96a9db477e7bb04537510273bd2 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5427974712386892261.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2491342391235333299.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3974451375754278496.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6272246368629475186.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3385975678059805383.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7307760640002470080.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5485297114194041187.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-28 06:00:30,839 1647c2a3 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/1647c2a3/pkb.log
2018-01-28 06:00:30,839 1647c2a3 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-28 06:00:30,840 1647c2a3 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-28 06:00:31,228 1647c2a3 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-28 06:00:31,228 1647c2a3 MainThread INFO Initializing the edw service decoder
2018-01-28 06:00:31,347 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-28 06:00:31,350 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-28 06:00:31,351 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-28 06:00:37,379 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-28 06:08:02,768 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-28 06:08:02,786 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1647c2a3/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-28 06:18:02,792 1647c2a3 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1647c2a3/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-28 06:18:02,799 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1647c2a3/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/1647c2a3/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/1647c2a3/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/1647c2a3/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-28 06:18:02,800 1647c2a3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-28 06:18:02,801 1647c2a3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-28 06:18:02,802 1647c2a3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-28 06:18:02,814 1647c2a3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-28 06:18:02,815 1647c2a3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/1647c2a3/pkb.log
2018-01-28 06:18:02,815 1647c2a3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/1647c2a3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #843
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/843/display/redirect?page=changes>
Changes:
[iemejia] Refactor code into idiomatic Java 8 style
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 29859eb54d05b96a9db477e7bb04537510273bd2 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 29859eb54d05b96a9db477e7bb04537510273bd2
Commit message: "Merge pull request #4511 from iemejia/idiomatic-java8"
> git rev-list b2e92836202ca32e90e9da3e1fd0fa09a0c781a7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8282686915856012349.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5129837539016255053.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8755681271830824686.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5650175455488390125.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8980161658220668370.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9213736558013139827.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8218243411704484962.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/77515e7b'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #842
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/842/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b2e92836202ca32e90e9da3e1fd0fa09a0c781a7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b2e92836202ca32e90e9da3e1fd0fa09a0c781a7
Commit message: "Merge pull request #4508 from iemejia/fix-java-modules"
> git rev-list b2e92836202ca32e90e9da3e1fd0fa09a0c781a7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1761225807258747139.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8215859036372273306.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3138864227032553154.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2616496331885007769.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins641923546490028677.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7562259234215813191.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6730452918807191241.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-27 18:00:25,648 ed684b01 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ed684b01/pkb.log
2018-01-27 18:00:25,648 ed684b01 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-27 18:00:25,649 ed684b01 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-27 18:00:25,839 ed684b01 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-27 18:00:25,840 ed684b01 MainThread INFO Initializing the edw service decoder
2018-01-27 18:00:26,005 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-27 18:00:26,009 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-27 18:00:26,010 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-27 18:00:32,940 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-27 18:08:18,660 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-27 18:08:18,677 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ed684b01/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-27 18:18:18,686 ed684b01 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ed684b01/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-27 18:18:18,693 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ed684b01/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ed684b01/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/ed684b01/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ed684b01/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-27 18:18:18,694 ed684b01 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-27 18:18:18,695 ed684b01 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-27 18:18:18,697 ed684b01 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-27 18:18:18,715 ed684b01 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-27 18:18:18,715 ed684b01 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ed684b01/pkb.log
2018-01-27 18:18:18,716 ed684b01 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ed684b01/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #841
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/841/display/redirect?page=changes>
Changes:
[iemejia] Fix modules that were activated only on Java 8 profile
[iemejia] [BEAM-3275] Fix ValidatesRunner Spark runner after the Kafka update
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b2e92836202ca32e90e9da3e1fd0fa09a0c781a7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b2e92836202ca32e90e9da3e1fd0fa09a0c781a7
Commit message: "Merge pull request #4508 from iemejia/fix-java-modules"
> git rev-list fec29b04b99976382993fbad79ae88ef44dad32a # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins504051579850477323.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9222222395947039553.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins792280374698522136.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2896625436563048738.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4321462714655828518.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins151525760708927599.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5331911729470733637.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-27 12:00:42,791 34c0d1a2 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/34c0d1a2/pkb.log
2018-01-27 12:00:42,792 34c0d1a2 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-27 12:00:42,792 34c0d1a2 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-27 12:00:43,521 34c0d1a2 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-27 12:00:43,521 34c0d1a2 MainThread INFO Initializing the edw service decoder
2018-01-27 12:00:43,651 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-27 12:00:43,654 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-27 12:00:43,654 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-27 12:00:49,472 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-27 12:09:12,664 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-27 12:09:12,690 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/34c0d1a2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-27 12:19:12,702 34c0d1a2 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/34c0d1a2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-27 12:19:12,710 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/34c0d1a2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/34c0d1a2/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/34c0d1a2/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/34c0d1a2/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-27 12:19:12,711 34c0d1a2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-27 12:19:12,712 34c0d1a2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-27 12:19:12,714 34c0d1a2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-27 12:19:12,752 34c0d1a2 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-27 12:19:12,753 34c0d1a2 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/34c0d1a2/pkb.log
2018-01-27 12:19:12,753 34c0d1a2 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/34c0d1a2/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #840
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/840/display/redirect?page=changes>
Changes:
[kirpichov] [BEAM-3083] Do not call getSchema() and getTable() on every element
[lcwik] [Beam-2500] Add S3FileSystem to SDKs/Java/IO
[lcwik] implement serializing AWS credentials provider
[lcwik] fixup! Clarify error message is received from SDK harness
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision fec29b04b99976382993fbad79ae88ef44dad32a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f fec29b04b99976382993fbad79ae88ef44dad32a
Commit message: "Merge pull request #4493 from jkff/bq-get-schema"
> git rev-list 71192743121d0b142828da870905369f15c64f15 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6110558747536243775.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3579965436926897140.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8464165503030377074.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4617725584641276885.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4941567669952889714.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7365563098356439566.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4918127418485034405.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/5b0ee54b'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #839
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/839/display/redirect?page=changes>
Changes:
[zoy] Increasing BatchElements's max_batch_size to 10K
[lcwik] [BEAM-2273] Cleanup examples Maven Archetype to copy in a clean state
[XuMingmin] [BEAM-3526] KakfaIO support for finalizeCheckpoint() (#4481)
[kedin] Add Avro dependency to KafkaIO
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 71192743121d0b142828da870905369f15c64f15 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 71192743121d0b142828da870905369f15c64f15
Commit message: "Merge pull request #4503: Add Avro dependency to KafkaIO"
> git rev-list d4a55d9f485a06a9fc0933884c36d38af3c69d3d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6456047027741313524.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1644363041463065565.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2048723133408611392.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4794801852816059528.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins855725714432159011.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3851115278688564585.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3421790190244657140.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/6dc0a5ec'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user XuMingmin@users.noreply.github.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user zoy@giggles.nyc.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #838
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/838/display/redirect?page=changes>
Changes:
[tgroh] Retrieve Environments from PTransforms
[iemejia] Remove unneeded profile for javadoc on Java 8
[iemejia] Remove unneeded explicit Java 8 references on maven-compiler-plugin
[iemejia] Fix doc error on hadoop-input-format ITs after move to Java 8 only tests
[iemejia] Remove references to non-existent examples:java8 module in gradle
[iemejia] Remove references to java 7/8 only examples from the README
[iemejia] Remove some comments on Java 7/8 only stuff that don't make sense
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision d4a55d9f485a06a9fc0933884c36d38af3c69d3d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f d4a55d9f485a06a9fc0933884c36d38af3c69d3d
Commit message: "Merge pull request #4469"
> git rev-list 87670e6f525f3a9e51f6603f072410f86be48447 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9122977290688440520.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins38554171917008379.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7999034665445874876.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins287907518671620106.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9121776821158601154.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2738712822857292674.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4208233546692201023.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/ac74bfbf'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #837
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/837/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 87670e6f525f3a9e51f6603f072410f86be48447 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 87670e6f525f3a9e51f6603f072410f86be48447
Commit message: "Merge pull request #4495: Use non-deprecated version of Futures.transform"
> git rev-list 87670e6f525f3a9e51f6603f072410f86be48447 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4712018958907391642.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2892737587523906551.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins437190026871285123.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5599878006945272551.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6604726722032393410.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8533897078964757765.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5094464750967335574.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/73ef9e2f'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #836
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/836/display/redirect?page=changes>
Changes:
[altay] Use non-deprecated version of Futures.transform
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 87670e6f525f3a9e51f6603f072410f86be48447 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 87670e6f525f3a9e51f6603f072410f86be48447
Commit message: "Merge pull request #4495: Use non-deprecated version of Futures.transform"
> git rev-list 0f554fd51414381646c32b36f6f87a14adc0fad8 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1998204094733082615.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8067254333195029362.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins961330500059049560.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7895692462601518943.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9203070869092914116.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4916248321347308564.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3225061134488231476.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/bc6ca451'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #835
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/835/display/redirect?page=changes>
Changes:
[shashank] fix serialization error in BigQueryIO's DynamicDestinations
[shashank] correct side input check in BigQueryIO DynamicDestination for pull
[coheigea] BEAM-3533 - Replace hard-coded UTF-8 Strings
[jbonofre] [BEAM-3466] Remove JDK 7 references in Jenkins
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0f554fd51414381646c32b36f6f87a14adc0fad8 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0f554fd51414381646c32b36f6f87a14adc0fad8
Commit message: "Merge pull request #4417 from shashank64/master"
> git rev-list c07a43ee2d4d81de15d78f8ec5a52816248d1afe # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8491686870287645687.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3740577391775067095.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3492718838635065982.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4433122496208090018.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6090587241159871283.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3553162801093382282.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7091985844151007933.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-26 00:00:41,180 8f18d952 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/8f18d952/pkb.log
2018-01-26 00:00:41,180 8f18d952 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-26 00:00:41,181 8f18d952 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-26 00:00:41,972 8f18d952 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-26 00:00:41,972 8f18d952 MainThread INFO Initializing the edw service decoder
2018-01-26 00:00:42,101 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-26 00:00:42,108 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-26 00:00:42,109 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-26 00:00:49,025 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-26 00:10:27,788 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-26 00:10:27,805 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8f18d952/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-26 00:20:27,811 8f18d952 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8f18d952/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-26 00:20:27,817 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8f18d952/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/8f18d952/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/8f18d952/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/8f18d952/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-26 00:20:27,818 8f18d952 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-26 00:20:27,818 8f18d952 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-26 00:20:27,820 8f18d952 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-26 00:20:27,862 8f18d952 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-26 00:20:27,862 8f18d952 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/8f18d952/pkb.log
2018-01-26 00:20:27,862 8f18d952 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/8f18d952/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #834
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/834/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3432] Remove hadoop/jdk1.8-tests module
[lcwik] [BEAM-3160] Fix issue where we would choose which coder to use
[jbonofre] [BEAM-3428] Merge Java8 examples in "main" Java examples
[jbonofre] [BEAM-3428] Replace MinimalWordCount with Java8 one
[echauchot] [BEAM-3534] Add a spark validates runner test for metrics sink in
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision c07a43ee2d4d81de15d78f8ec5a52816248d1afe (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f c07a43ee2d4d81de15d78f8ec5a52816248d1afe
Commit message: "Merge pull request #4433: [BEAM-3160] Prevent issue where we would choose which coder to use arbitrarily when it is over specified with multiple coders"
> git rev-list ad82ac140161cd5d88c972c88a1c8168cf26ff1c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1965223259982212590.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2271692636030925425.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4461281613112259064.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6067582894971089092.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins143924699332470949.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6472190206500764377.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1775861125409526115.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-25 18:00:38,083 391513ea MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/391513ea/pkb.log
2018-01-25 18:00:38,083 391513ea MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-25 18:00:38,084 391513ea MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-25 18:00:38,413 391513ea MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-25 18:00:38,413 391513ea MainThread INFO Initializing the edw service decoder
2018-01-25 18:00:38,559 391513ea MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-25 18:00:38,562 391513ea MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-25 18:00:38,562 391513ea MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-25 18:00:45,094 391513ea MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-25 18:07:15,908 391513ea MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-25 18:07:15,922 391513ea MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/391513ea/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-25 18:17:15,928 391513ea Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/391513ea/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-25 18:17:15,934 391513ea MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/391513ea/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/391513ea/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/391513ea/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/391513ea/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-25 18:17:15,935 391513ea MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-25 18:17:15,936 391513ea MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-25 18:17:15,938 391513ea MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-25 18:17:15,964 391513ea MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-25 18:17:15,965 391513ea MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/391513ea/pkb.log
2018-01-25 18:17:15,965 391513ea MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/391513ea/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #833
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/833/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ad82ac140161cd5d88c972c88a1c8168cf26ff1c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ad82ac140161cd5d88c972c88a1c8168cf26ff1c
Commit message: "[BEAM-3060] Support for Perfkit execution of file-based-io-tests on HDFS cluster. (#4401)"
> git rev-list ad82ac140161cd5d88c972c88a1c8168cf26ff1c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1334120425936805997.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3859997261756042301.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1787890406216042711.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5609685352209704234.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1949501763166778118.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3681166176762255224.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6110439637883638970.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-25 12:00:49,829 62a3de6c MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/62a3de6c/pkb.log
2018-01-25 12:00:49,829 62a3de6c MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-25 12:00:49,830 62a3de6c MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-25 12:00:50,214 62a3de6c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-25 12:00:50,214 62a3de6c MainThread INFO Initializing the edw service decoder
2018-01-25 12:00:50,360 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-25 12:00:50,362 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-25 12:00:50,363 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-25 12:00:56,581 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-25 12:08:04,778 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-25 12:08:04,793 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/62a3de6c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-25 12:18:04,812 62a3de6c Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/62a3de6c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-25 12:18:04,821 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/62a3de6c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/62a3de6c/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/62a3de6c/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/62a3de6c/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-25 12:18:04,822 62a3de6c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-25 12:18:04,823 62a3de6c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-25 12:18:04,826 62a3de6c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-25 12:18:04,893 62a3de6c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-25 12:18:04,893 62a3de6c MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/62a3de6c/pkb.log
2018-01-25 12:18:04,893 62a3de6c MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/62a3de6c/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #832
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/832/display/redirect?page=changes>
Changes:
[chamikara] [BEAM-3060] Support for Perfkit execution of file-based-io-tests on HDFS
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ad82ac140161cd5d88c972c88a1c8168cf26ff1c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ad82ac140161cd5d88c972c88a1c8168cf26ff1c
Commit message: "[BEAM-3060] Support for Perfkit execution of file-based-io-tests on HDFS cluster. (#4401)"
> git rev-list 3c6899c001b28ff98db461df8b712c691403b67d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3446943244834405499.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1201958968145959151.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins264380244794727142.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins354247159521744660.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins954011160583374118.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9175755343204911643.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3724021084561376564.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-25 06:00:40,810 9698887e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/9698887e/pkb.log
2018-01-25 06:00:40,810 9698887e MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-25 06:00:40,811 9698887e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-25 06:00:41,236 9698887e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-25 06:00:41,236 9698887e MainThread INFO Initializing the edw service decoder
2018-01-25 06:00:41,378 9698887e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-25 06:00:41,381 9698887e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-25 06:00:41,382 9698887e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-25 06:00:47,397 9698887e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-25 06:07:58,296 9698887e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-25 06:07:58,309 9698887e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9698887e/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-25 06:17:58,314 9698887e Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9698887e/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-25 06:17:58,319 9698887e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9698887e/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9698887e/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/9698887e/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9698887e/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-25 06:17:58,319 9698887e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-25 06:17:58,320 9698887e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-25 06:17:58,322 9698887e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-25 06:17:58,348 9698887e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-25 06:17:58,349 9698887e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/9698887e/pkb.log
2018-01-25 06:17:58,349 9698887e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/9698887e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #831
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/831/display/redirect?page=changes>
Changes:
[sduskis] [BEAM-3412] Upgrade Cloud Bigtable to 1.0.0 The 1.0.0 Cloud Bigtable
[sduskis] BigtableServiceImplTest now uses a List of ranges. The PR did not
[sduskis] Updating Cloud Bigtable dependnecy to 1.0.0 in build.gradle.
[robert] Add CONTRIBUTING.md
[sduskis] Using 1.0.0-pre3 for bigtable-proto. This should allow Cloud Bigtable's
[sduskis] Updating build.gradle with a bigtable_proto_version of 1.0.0-pre3
[rober] Update printed gcloud cancel commands to include the job's region.
[sduskis] Fixing a bad merge in BigtableServiceImpl.
[kirpichov] Code compiles after auto-transition to lambda
[kirpichov] google-java-format
[kirpichov] Removes unnecessary explicit type arguments
[kirpichov] google-java-format
[kirpichov] checkstyle fixups
[kirpichov] More removal of explicit type arguments
[kirpichov] google-java-format
[kirpichov] Manually fixed a couple cases of bad comment formatting
[kirpichov] Manual fixup of some call sites where lambdas mess up coder inference
[kirpichov] A couple of final example fixups
[lcwik] Upgrade Jenkins jobs to use Maven 3.5.2
[lcwik] [BEAM-2728] Add Count-Min Sketch in sketching extension
[lcwik] Small fix in SketchCoder
[lcwik] Make Sketch AutoValue + Javadoc update
[lcwik] Optimize coder memory use
[lcwik] Initial post-release snapshot test
[lcwik] Make the snapshot and url parameters to the job to allow installing a
[lcwik] Make the snapshot and url parameters to the job to allow installing a
[lcwik] Rename TestHelper to TestScripts, it() to intent()
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c6899c001b28ff98db461df8b712c691403b67d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c6899c001b28ff98db461df8b712c691403b67d
Commit message: "Rename TestHelper to TestScripts, it() to intent()"
> git rev-list e70b0688fa57ad381a97630710c75233a4bb72f7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8122958545394056979.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4966290593879244958.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5270817778585768134.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8655215785668634611.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6726198599714045848.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8958012337260214830.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6925420453813369173.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/bb2e128d'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user sduskis@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user robert@frantil.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user rober@frantil.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #830
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/830/display/redirect?page=changes>
Changes:
[tgroh] Move BeamFnDataInboundObserver to java-fn-execution
[tgroh] Add an implementation of the Fn API Data Service
[zhouhai02] Update copyright date in NOTICE
[robertwb] [BEAM-3126] Adding a new Flatten test to Python SDK. (#4463)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e70b0688fa57ad381a97630710c75233a4bb72f7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e70b0688fa57ad381a97630710c75233a4bb72f7
Commit message: "[BEAM-3126] Adding a new Flatten test to Python SDK. (#4463)"
> git rev-list 57e8525a3c5514e10a746c32a5026505c05ca89a # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7234223143109049168.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3119871768677111673.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5343225476585477848.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4629996982164674977.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7746025601693731525.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3853171840260729418.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2866978486912076783.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/1b96d3c5'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user zhouhai02@meituan.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #829
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/829/display/redirect?page=changes>
Changes:
[lcwik] Migrate Flink ValidatesRunner to Gradle
[lcwik] Increment the Dataflow runner major version to 7.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 57e8525a3c5514e10a746c32a5026505c05ca89a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 57e8525a3c5514e10a746c32a5026505c05ca89a
Commit message: "Increment the Dataflow runner major version to 7."
> git rev-list f2503bad7511ef5f4856fc8af9c24b01a8561b3c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7133129044112003529.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2237047260159853846.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8925773676588600995.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9138457632051286981.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7976257638749750357.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3851041847596257574.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3619341336458645711.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-24 12:01:17,497 dcaedd7f MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/dcaedd7f/pkb.log
2018-01-24 12:01:17,497 dcaedd7f MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-24 12:01:17,498 dcaedd7f MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-24 12:01:17,899 dcaedd7f MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-24 12:01:17,899 dcaedd7f MainThread INFO Initializing the edw service decoder
2018-01-24 12:01:18,015 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-24 12:01:18,017 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-24 12:01:18,018 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-24 12:01:25,344 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-24 12:12:07,612 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-24 12:12:07,626 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/dcaedd7f/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-24 12:22:07,630 dcaedd7f Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/dcaedd7f/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-24 12:22:07,637 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/dcaedd7f/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/dcaedd7f/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/dcaedd7f/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/dcaedd7f/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-24 12:22:07,638 dcaedd7f MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-24 12:22:07,640 dcaedd7f MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-24 12:22:07,644 dcaedd7f MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-24 12:22:07,685 dcaedd7f MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-24 12:22:07,685 dcaedd7f MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/dcaedd7f/pkb.log
2018-01-24 12:22:07,686 dcaedd7f MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/dcaedd7f/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #828
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/828/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-3008] Adds parameters templatization for Bigtable (#4357)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2503bad7511ef5f4856fc8af9c24b01a8561b3c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f2503bad7511ef5f4856fc8af9c24b01a8561b3c
Commit message: "[BEAM-3008] Adds parameters templatization for Bigtable (#4357)"
> git rev-list 6663e6038ca4293ecb65627e30bca6e41556e6ff # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4234987051030053190.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1498415613070168989.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4972741531330052470.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins959434350845561699.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6433174149059762558.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3451380411316575575.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2812178036413106879.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-24 06:00:35,941 dc3ded26 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/dc3ded26/pkb.log
2018-01-24 06:00:35,941 dc3ded26 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-24 06:00:35,942 dc3ded26 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-24 06:00:36,234 dc3ded26 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-24 06:00:36,234 dc3ded26 MainThread INFO Initializing the edw service decoder
2018-01-24 06:00:36,382 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-24 06:00:36,384 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-24 06:00:36,385 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-24 06:00:42,073 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-24 06:07:55,082 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-24 06:07:55,095 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/dc3ded26/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-24 06:17:55,099 dc3ded26 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/dc3ded26/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-24 06:17:55,104 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/dc3ded26/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/dc3ded26/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/dc3ded26/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/dc3ded26/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-24 06:17:55,105 dc3ded26 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-24 06:17:55,105 dc3ded26 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-24 06:17:55,107 dc3ded26 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-24 06:17:55,152 dc3ded26 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-24 06:17:55,153 dc3ded26 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/dc3ded26/pkb.log
2018-01-24 06:17:55,153 dc3ded26 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/dc3ded26/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #827
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/827/display/redirect?page=changes>
Changes:
[github] Fix code comment to match code
[github] NotImplementedErrror --> NotImplementedError
[robertwb] Remove legacy windowfn translation.
[github] Update BUILD.md
[batbat] Added an example pipeline that uses stateful processing to output team
[robertwb] Manually specify direct runner for global-state modifying tests.
[robertwb] FakeSource requires direct runner.
[robertwb] Explicitly use DirectRunner in DirectRunner tests.
[altay] Disable combiner lifting optimization in DataflowRunner for fnapi
[ehudm] Pass PipelineOptions to FileSystem constructor.
[altay] Disable combiner lifting when only the streaming flag is set.
[tgroh] Register Environments in SdkComponents
[robertwb] Document DirectRunnerOnly tests.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 6663e6038ca4293ecb65627e30bca6e41556e6ff (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6663e6038ca4293ecb65627e30bca6e41556e6ff
Commit message: "Merge pull request #4465 from udim/filesystem-options"
> git rev-list 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5335031508465390845.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5320057679787662314.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9088297080184699183.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4743533065908698620.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8164324441801399250.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7214718998816030152.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8001308199063174263.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/6d9804e6'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #826
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/826/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0
Commit message: "Merge pull request #4437 from robertwb/runner-api-combine-fn"
> git rev-list 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1023015293746055132.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8502518211590891455.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4638467709473828950.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6761367883247122626.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3637916690236816226.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4048195739762904540.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3295469204381942815.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-23 18:00:30,816 f9ab263c MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/f9ab263c/pkb.log
2018-01-23 18:00:30,816 f9ab263c MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-23 18:00:30,817 f9ab263c MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-23 18:00:31,101 f9ab263c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-23 18:00:31,101 f9ab263c MainThread INFO Initializing the edw service decoder
2018-01-23 18:00:31,224 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-23 18:00:31,227 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-23 18:00:31,228 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-23 18:00:37,152 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-23 18:07:20,191 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-23 18:07:20,205 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f9ab263c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-23 18:17:20,210 f9ab263c Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f9ab263c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-23 18:17:20,218 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f9ab263c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f9ab263c/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/f9ab263c/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f9ab263c/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-23 18:17:20,218 f9ab263c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-23 18:17:20,250 f9ab263c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-23 18:17:20,252 f9ab263c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-23 18:17:20,273 f9ab263c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-23 18:17:20,273 f9ab263c MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f9ab263c/pkb.log
2018-01-23 18:17:20,273 f9ab263c MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f9ab263c/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #825
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/825/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0
Commit message: "Merge pull request #4437 from robertwb/runner-api-combine-fn"
> git rev-list 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5693644691355979568.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1345450801246266482.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3269586146178397808.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1422334400827537723.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5841384343184748154.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7924711260756307999.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8025938385309317919.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-23 12:00:29,309 37c5ae36 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/37c5ae36/pkb.log
2018-01-23 12:00:29,309 37c5ae36 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-23 12:00:29,310 37c5ae36 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-23 12:00:29,595 37c5ae36 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-23 12:00:29,596 37c5ae36 MainThread INFO Initializing the edw service decoder
2018-01-23 12:00:29,737 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-23 12:00:29,740 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-23 12:00:29,741 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-23 12:00:35,543 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-23 12:07:28,418 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-23 12:07:28,432 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/37c5ae36/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-23 12:17:28,436 37c5ae36 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/37c5ae36/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-23 12:17:28,453 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/37c5ae36/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/37c5ae36/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/37c5ae36/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/37c5ae36/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-23 12:17:28,454 37c5ae36 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-23 12:17:28,456 37c5ae36 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-23 12:17:28,460 37c5ae36 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-23 12:17:28,483 37c5ae36 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-23 12:17:28,485 37c5ae36 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/37c5ae36/pkb.log
2018-01-23 12:17:28,485 37c5ae36 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/37c5ae36/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #824
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/824/display/redirect?page=changes>
Changes:
[robertwb] Curry CombineFn arguments into runner API protos.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 94d5c0b71fa3017c855f59ced6c94fa5c7c735e0
Commit message: "Merge pull request #4437 from robertwb/runner-api-combine-fn"
> git rev-list 55844fa6c53ae38627d143eede7d5781984621c3 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3077731813704183319.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4620926538004712425.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins364794308052023614.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3177191037813556367.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1188801084505075078.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8658029701141375978.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7526873909890689873.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-23 06:00:30,929 96f1fe36 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/96f1fe36/pkb.log
2018-01-23 06:00:30,929 96f1fe36 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-23 06:00:30,930 96f1fe36 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-23 06:00:31,323 96f1fe36 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-23 06:00:31,324 96f1fe36 MainThread INFO Initializing the edw service decoder
2018-01-23 06:00:31,477 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-23 06:00:31,480 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-23 06:00:31,480 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-23 06:00:37,133 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-23 06:07:48,414 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-23 06:07:48,427 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/96f1fe36/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-23 06:17:48,432 96f1fe36 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/96f1fe36/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-23 06:17:48,437 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/96f1fe36/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/96f1fe36/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/96f1fe36/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/96f1fe36/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-23 06:17:48,438 96f1fe36 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-23 06:17:48,439 96f1fe36 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-23 06:17:48,441 96f1fe36 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-23 06:17:48,498 96f1fe36 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-23 06:17:48,498 96f1fe36 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/96f1fe36/pkb.log
2018-01-23 06:17:48,498 96f1fe36 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/96f1fe36/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #823
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/823/display/redirect?page=changes>
Changes:
[markliu] [BEAM-2762] Python code coverage report in Postcommit
[markliu] fixup! Exclude auto-generated files in portability/api
[markliu] fixup! Clean up configuration and use better way to exclude cover in mvn
[markliu] fixup! Add more comments
[tgroh] Add a BundleProcessor to SdkHarnessClient
[robertwb] [BEAM-3490] Picklable side inputs for FnApi Workers.
[markliu] fixup! Improve comments
[github] [BEAM-2963] Remove layer of indirection in output name mapping in
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 55844fa6c53ae38627d143eede7d5781984621c3 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 55844fa6c53ae38627d143eede7d5781984621c3
Commit message: "Merge pull request #4266"
> git rev-list 53e4b938fe0064e8e3dc5a5563488ad7dbe6a731 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7940650052058123712.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7357478019487794645.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7225453636560085565.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8700454780521826328.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4724508750962531586.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8166140423735541561.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7813896127891253526.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-23 00:00:33,845 9a72a0a6 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/9a72a0a6/pkb.log
2018-01-23 00:00:33,845 9a72a0a6 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-23 00:00:33,846 9a72a0a6 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-23 00:00:34,131 9a72a0a6 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-23 00:00:34,132 9a72a0a6 MainThread INFO Initializing the edw service decoder
2018-01-23 00:00:34,259 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-23 00:00:34,262 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-23 00:00:34,263 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-23 00:00:40,397 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-23 00:08:03,891 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-23 00:08:03,905 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9a72a0a6/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-23 00:18:03,910 9a72a0a6 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9a72a0a6/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-23 00:18:03,916 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9a72a0a6/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9a72a0a6/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/9a72a0a6/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9a72a0a6/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-23 00:18:03,916 9a72a0a6 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-23 00:18:03,917 9a72a0a6 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-23 00:18:03,919 9a72a0a6 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-23 00:18:03,964 9a72a0a6 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-23 00:18:03,965 9a72a0a6 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/9a72a0a6/pkb.log
2018-01-23 00:18:03,965 9a72a0a6 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/9a72a0a6/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user markliu@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #822
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/822/display/redirect?page=changes>
Changes:
[jb] [BEAM-3507] Add a way to specify the batch size in JdbcIO Write
[jbonofre] [BEAM-3507] Add DEFAULT_BATCH_SIZE and use batchSize in default
[iemejia] [BEAM-3492] Fix spark 2.10 dependency leak on hadoop tests
[iemejia] [BEAM-3492] Force netty version when running with the spark runner
[lcwik] [BEAM-3427] Update build to Java 8 (before clean out) (#4424)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 53e4b938fe0064e8e3dc5a5563488ad7dbe6a731 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 53e4b938fe0064e8e3dc5a5563488ad7dbe6a731
Commit message: "Merge pull request #4459 from iemejia/BEAM-3492-spark-e2e-wordcount"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4167042280217033993.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8181285733688941764.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins238889367944964521.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7158681242073200858.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4690312985613562885.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5262530910786062658.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8338295999364098692.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-22 18:00:39,859 a9b988dc MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/a9b988dc/pkb.log
2018-01-22 18:00:39,860 a9b988dc MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-22 18:00:39,860 a9b988dc MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-22 18:00:40,275 a9b988dc MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-22 18:00:40,276 a9b988dc MainThread INFO Initializing the edw service decoder
2018-01-22 18:00:40,542 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-22 18:00:40,546 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-22 18:00:40,548 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-22 18:00:48,545 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-22 18:08:44,044 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-22 18:08:44,091 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a9b988dc/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-22 18:18:44,096 a9b988dc Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a9b988dc/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-22 18:18:44,104 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a9b988dc/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a9b988dc/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/a9b988dc/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a9b988dc/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-22 18:18:44,105 a9b988dc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-22 18:18:44,106 a9b988dc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-22 18:18:44,107 a9b988dc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-22 18:18:44,137 a9b988dc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-22 18:18:44,138 a9b988dc MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/a9b988dc/pkb.log
2018-01-22 18:18:44,138 a9b988dc MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/a9b988dc/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #821
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/821/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3886748865992429641.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins934296149826016086.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6664307546947651513.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4991986986191272283.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5858694325358033254.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins363108574223702102.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4489270469424408061.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-22 12:00:38,029 af509f0e MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/af509f0e/pkb.log
2018-01-22 12:00:38,029 af509f0e MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-22 12:00:38,030 af509f0e MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-22 12:00:38,710 af509f0e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-22 12:00:38,711 af509f0e MainThread INFO Initializing the edw service decoder
2018-01-22 12:00:38,907 af509f0e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-22 12:00:38,910 af509f0e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-22 12:00:38,911 af509f0e MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-22 12:00:45,089 af509f0e MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-22 12:07:51,925 af509f0e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-22 12:07:51,939 af509f0e MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/af509f0e/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-22 12:17:51,944 af509f0e Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/af509f0e/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-22 12:17:51,951 af509f0e MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/af509f0e/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/af509f0e/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/af509f0e/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/af509f0e/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-22 12:17:51,962 af509f0e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-22 12:17:51,964 af509f0e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-22 12:17:51,966 af509f0e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-22 12:17:51,989 af509f0e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-22 12:17:51,990 af509f0e MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/af509f0e/pkb.log
2018-01-22 12:17:51,990 af509f0e MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/af509f0e/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #820
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/820/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3389822559119159592.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7551134456365099959.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7833773733442154936.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7938299150020861914.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3854479505894261881.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4918878649756010139.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7457961412158614992.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-22 06:00:27,796 3acba496 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/3acba496/pkb.log
2018-01-22 06:00:27,797 3acba496 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-22 06:00:27,797 3acba496 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-22 06:00:28,042 3acba496 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-22 06:00:28,043 3acba496 MainThread INFO Initializing the edw service decoder
2018-01-22 06:00:28,176 3acba496 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-22 06:00:28,179 3acba496 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-22 06:00:28,180 3acba496 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-22 06:00:34,162 3acba496 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-22 06:07:03,971 3acba496 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-22 06:07:03,987 3acba496 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3acba496/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-22 06:17:03,993 3acba496 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3acba496/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-22 06:17:04,001 3acba496 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3acba496/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/3acba496/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/3acba496/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/3acba496/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-22 06:17:04,002 3acba496 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-22 06:17:04,003 3acba496 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-22 06:17:04,006 3acba496 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-22 06:17:04,045 3acba496 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-22 06:17:04,045 3acba496 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/3acba496/pkb.log
2018-01-22 06:17:04,045 3acba496 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/3acba496/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #819
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/819/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1103386982507548433.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins963932734629361957.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1032539168419366954.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7933873394642469311.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3080588526702673187.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6819530760375305487.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2398976843693891467.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-22 00:00:22,704 846c09fe MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/846c09fe/pkb.log
2018-01-22 00:00:22,704 846c09fe MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-22 00:00:22,704 846c09fe MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-22 00:00:22,992 846c09fe MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-22 00:00:22,993 846c09fe MainThread INFO Initializing the edw service decoder
2018-01-22 00:00:23,124 846c09fe MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-22 00:00:23,127 846c09fe MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-22 00:00:23,128 846c09fe MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-22 00:00:29,978 846c09fe MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-22 00:07:11,888 846c09fe MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-22 00:07:11,904 846c09fe MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/846c09fe/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-22 00:17:11,911 846c09fe Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/846c09fe/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-22 00:17:11,917 846c09fe MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/846c09fe/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/846c09fe/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/846c09fe/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/846c09fe/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-22 00:17:11,918 846c09fe MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-22 00:17:11,919 846c09fe MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-22 00:17:11,922 846c09fe MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-22 00:17:11,972 846c09fe MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-22 00:17:11,973 846c09fe MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/846c09fe/pkb.log
2018-01-22 00:17:11,973 846c09fe MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/846c09fe/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #818
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/818/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1899803924623333056.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6008380351453961158.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins615430612612309621.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6680365037261653027.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8646777932310859487.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5942034196655652127.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4272064868333075985.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-21 18:00:29,596 b6ad8c60 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/b6ad8c60/pkb.log
2018-01-21 18:00:29,596 b6ad8c60 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-21 18:00:29,597 b6ad8c60 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-21 18:00:30,013 b6ad8c60 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-21 18:00:30,014 b6ad8c60 MainThread INFO Initializing the edw service decoder
2018-01-21 18:00:30,137 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-21 18:00:30,140 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-21 18:00:30,141 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-21 18:00:36,345 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-21 18:07:37,045 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-21 18:07:37,059 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b6ad8c60/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-21 18:17:37,064 b6ad8c60 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b6ad8c60/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-21 18:17:37,069 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/b6ad8c60/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/b6ad8c60/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/b6ad8c60/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/b6ad8c60/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-21 18:17:37,069 b6ad8c60 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-21 18:17:37,070 b6ad8c60 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-21 18:17:37,072 b6ad8c60 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-21 18:17:37,085 b6ad8c60 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-21 18:17:37,085 b6ad8c60 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/b6ad8c60/pkb.log
2018-01-21 18:17:37,085 b6ad8c60 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/b6ad8c60/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #817
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/817/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins992739342341760396.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6602389837240712415.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6883182215350353702.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8770700351914146735.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins983561420020702950.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3976963997230496742.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5226491658153342715.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-21 12:00:28,746 9198fccf MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/9198fccf/pkb.log
2018-01-21 12:00:28,746 9198fccf MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-21 12:00:28,746 9198fccf MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-21 12:00:29,169 9198fccf MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-21 12:00:29,170 9198fccf MainThread INFO Initializing the edw service decoder
2018-01-21 12:00:29,311 9198fccf MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-21 12:00:29,314 9198fccf MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-21 12:00:29,315 9198fccf MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-21 12:00:36,032 9198fccf MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-21 12:07:35,555 9198fccf MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-21 12:07:35,570 9198fccf MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9198fccf/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-21 12:17:35,574 9198fccf Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9198fccf/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-21 12:17:35,580 9198fccf MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9198fccf/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9198fccf/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/9198fccf/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9198fccf/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-21 12:17:35,580 9198fccf MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-21 12:17:35,581 9198fccf MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-21 12:17:35,583 9198fccf MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-21 12:17:35,599 9198fccf MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-21 12:17:35,600 9198fccf MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/9198fccf/pkb.log
2018-01-21 12:17:35,600 9198fccf MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/9198fccf/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #816
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/816/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2103953070590454921.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3652653394558591297.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins24908017030323230.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins60925262519150802.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4645726033393306869.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2458780223506129431.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8117054554505034742.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-21 06:00:30,027 15471219 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/15471219/pkb.log
2018-01-21 06:00:30,027 15471219 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-21 06:00:30,028 15471219 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-21 06:00:30,414 15471219 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-21 06:00:30,414 15471219 MainThread INFO Initializing the edw service decoder
2018-01-21 06:00:30,533 15471219 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-21 06:00:30,536 15471219 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-21 06:00:30,537 15471219 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-21 06:00:36,470 15471219 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-21 06:07:46,832 15471219 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-21 06:07:46,847 15471219 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/15471219/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-21 06:17:46,852 15471219 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/15471219/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-21 06:17:46,858 15471219 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/15471219/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/15471219/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/15471219/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/15471219/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-21 06:17:46,859 15471219 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-21 06:17:46,860 15471219 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-21 06:17:46,863 15471219 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-21 06:17:46,904 15471219 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-21 06:17:46,905 15471219 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/15471219/pkb.log
2018-01-21 06:17:46,905 15471219 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/15471219/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #815
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/815/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins53124542827277044.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5089179706907421137.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins548078592928898209.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1232089252885700947.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8069885734423739035.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins679679032232831682.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins290224375730121225.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-21 00:00:42,258 9e6ac1b5 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/9e6ac1b5/pkb.log
2018-01-21 00:00:42,258 9e6ac1b5 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-21 00:00:42,259 9e6ac1b5 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-21 00:00:42,813 9e6ac1b5 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-21 00:00:42,814 9e6ac1b5 MainThread INFO Initializing the edw service decoder
2018-01-21 00:00:42,992 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-21 00:00:42,995 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-21 00:00:42,995 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-21 00:00:50,796 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-21 00:08:10,635 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-21 00:08:10,649 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9e6ac1b5/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-21 00:18:10,654 9e6ac1b5 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9e6ac1b5/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-21 00:18:10,664 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9e6ac1b5/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9e6ac1b5/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/9e6ac1b5/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9e6ac1b5/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-21 00:18:10,665 9e6ac1b5 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-21 00:18:10,667 9e6ac1b5 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-21 00:18:10,670 9e6ac1b5 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-21 00:18:10,688 9e6ac1b5 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-21 00:18:10,689 9e6ac1b5 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/9e6ac1b5/pkb.log
2018-01-21 00:18:10,689 9e6ac1b5 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/9e6ac1b5/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #814
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/814/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5952477558570917013.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins363362656116482434.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8410161024090928041.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6361067148810040934.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6457649482888663366.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins377824347689662342.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins610822174296003100.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-20 18:00:22,637 8b1f89dd MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/8b1f89dd/pkb.log
2018-01-20 18:00:22,638 8b1f89dd MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-20 18:00:22,638 8b1f89dd MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-20 18:00:23,031 8b1f89dd MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-20 18:00:23,031 8b1f89dd MainThread INFO Initializing the edw service decoder
2018-01-20 18:00:23,151 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-20 18:00:23,154 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-20 18:00:23,154 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-20 18:00:28,908 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-20 18:07:15,771 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-20 18:07:15,790 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8b1f89dd/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-20 18:17:15,795 8b1f89dd Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8b1f89dd/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-20 18:17:15,802 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/8b1f89dd/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/8b1f89dd/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/8b1f89dd/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/8b1f89dd/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-20 18:17:15,803 8b1f89dd MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-20 18:17:15,862 8b1f89dd MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-20 18:17:15,865 8b1f89dd MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-20 18:17:15,921 8b1f89dd MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-20 18:17:15,922 8b1f89dd MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/8b1f89dd/pkb.log
2018-01-20 18:17:15,922 8b1f89dd MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/8b1f89dd/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #813
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/813/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 3c4a2001e5364657f6596bfc338f792e4797712d # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1338655540356708607.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3743866422635194522.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7613268411817791959.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1191180950550401830.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7467822004179424929.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7864885383848279969.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8750568491056100016.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-20 12:00:30,845 0f663d5a MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/0f663d5a/pkb.log
2018-01-20 12:00:30,845 0f663d5a MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-20 12:00:30,846 0f663d5a MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-20 12:00:31,423 0f663d5a MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-20 12:00:31,424 0f663d5a MainThread INFO Initializing the edw service decoder
2018-01-20 12:00:31,541 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-20 12:00:31,544 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-20 12:00:31,545 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-20 12:00:37,416 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-20 12:07:41,704 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-20 12:07:41,720 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0f663d5a/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-20 12:17:41,725 0f663d5a Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0f663d5a/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-20 12:17:41,730 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/0f663d5a/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/0f663d5a/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/0f663d5a/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/0f663d5a/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-20 12:17:41,731 0f663d5a MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-20 12:17:41,732 0f663d5a MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-20 12:17:41,734 0f663d5a MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-20 12:17:41,753 0f663d5a MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-20 12:17:41,754 0f663d5a MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/0f663d5a/pkb.log
2018-01-20 12:17:41,754 0f663d5a MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/0f663d5a/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #812
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/812/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
> git rev-list 898277297be7160900fdb48ed486a257a6bc92c0 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
java.util.concurrent.TimeoutException: Timeout waiting for task.
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:259)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:91)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.atlassian.jira.rest.client.internal.async.DelegatingPromise.get(DelegatingPromise.java:107)
at hudson.plugins.jira.JiraRestService.getIssuesFromJqlSearch(JiraRestService.java:177)
at hudson.plugins.jira.JiraSession.getIssuesFromJqlSearch(JiraSession.java:135)
at io.jenkins.blueocean.service.embedded.jira.JiraSCMListener.onChangeLogParsed(JiraSCMListener.java:43)
at hudson.model.listeners.SCMListener.onChangeLogParsed(SCMListener.java:120)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:590)
Caused: java.io.IOException: Failed to parse changelog
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:592)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
java.util.concurrent.TimeoutException: Timeout waiting for task.
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:259)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:91)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.atlassian.jira.rest.client.internal.async.DelegatingPromise.get(DelegatingPromise.java:107)
at hudson.plugins.jira.JiraRestService.getIssuesFromJqlSearch(JiraRestService.java:177)
at hudson.plugins.jira.JiraSession.getIssuesFromJqlSearch(JiraSession.java:135)
at io.jenkins.blueocean.service.embedded.jira.JiraSCMListener.onChangeLogParsed(JiraSCMListener.java:43)
at hudson.model.listeners.SCMListener.onChangeLogParsed(SCMListener.java:120)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:590)
Caused: java.io.IOException: Failed to parse changelog
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:592)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Retrying after 10 seconds
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3c4a2001e5364657f6596bfc338f792e4797712d (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3c4a2001e5364657f6596bfc338f792e4797712d
Commit message: "[BEAM-3502] Remove usage of proto.Builder.clone() in DatastoreIO (#4449)"
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
java.util.concurrent.TimeoutException: Timeout waiting for task.
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:259)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:91)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.atlassian.jira.rest.client.internal.async.DelegatingPromise.get(DelegatingPromise.java:107)
at hudson.plugins.jira.JiraRestService.getIssuesFromJqlSearch(JiraRestService.java:177)
at hudson.plugins.jira.JiraSession.getIssuesFromJqlSearch(JiraSession.java:135)
at io.jenkins.blueocean.service.embedded.jira.JiraSCMListener.onChangeLogParsed(JiraSCMListener.java:43)
at hudson.model.listeners.SCMListener.onChangeLogParsed(SCMListener.java:120)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:590)
Caused: java.io.IOException: Failed to parse changelog
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:592)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #811
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/811/display/redirect?page=changes>
Changes:
[batbat] Moved floating point equality findbugs annotation from generic xml file
[melissapa] [BEAM-3351] Fix Javadoc formatting issues
[melissapa] Update Context references to links
[tgroh] Implement a GRPC Provision Service
[tgroh] Add InboundDataClient
[lcwik] Use platformThreadFactory for default thread pool.
[robertwb] [BEAM-3490] Wrap DistributionData in a DistributionResult for
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 898277297be7160900fdb48ed486a257a6bc92c0 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 898277297be7160900fdb48ed486a257a6bc92c0
Commit message: "Merge pull request #4450 [BEAM-3490] Wrap DistributionData in a DistributionResult"
> git rev-list a91f7ada2075b8d6adbbb7002a15f1078d8eae45 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5835360861966226757.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1437222341222237480.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4062026425642829119.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5221037234891501694.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6219418177846044991.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1362464057200307712.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4909729461215156437.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/9490dee9'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user melissapa@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #810
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/810/display/redirect?page=changes>
Changes:
[holdenkarau] Apply futurize on gen_protos
[holdenkarau] Run futurize regardless
[holdenkarau] Only futurize for sys version geq 3
[holdenkarau] Long form command line options
[iemejia] [BEAM-3275] Update KafkaIO to support Kafka version 1.0.0
[lcwik] Update generated version of Fn API code.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a91f7ada2075b8d6adbbb7002a15f1078d8eae45 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a91f7ada2075b8d6adbbb7002a15f1078d8eae45
Commit message: "Merge pull request #4077 from holdenk/BEAM-3142-fix-proto-gen-for-py3"
> git rev-list a7f4e56cdd47fd7432cd70e9897fdc18d40eb74c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4617194359436088161.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1054359612651489722.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins645331464544131530.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins64471939030947717.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4634921484544678205.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5688904028920415625.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3497628454461741982.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-19 18:00:38,111 a6f9e136 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/a6f9e136/pkb.log
2018-01-19 18:00:38,114 a6f9e136 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-19 18:00:38,114 a6f9e136 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-19 18:00:38,495 a6f9e136 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-19 18:00:38,496 a6f9e136 MainThread INFO Initializing the edw service decoder
2018-01-19 18:00:38,654 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-19 18:00:38,658 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-19 18:00:38,659 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-19 18:00:46,897 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-19 18:12:16,513 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-19 18:12:16,743 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a6f9e136/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-19 18:22:16,788 a6f9e136 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a6f9e136/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-19 18:22:16,796 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a6f9e136/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a6f9e136/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/a6f9e136/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a6f9e136/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-19 18:22:16,797 a6f9e136 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-19 18:22:16,860 a6f9e136 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-19 18:22:16,863 a6f9e136 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-19 18:22:16,935 a6f9e136 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-19 18:22:16,936 a6f9e136 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/a6f9e136/pkb.log
2018-01-19 18:22:16,936 a6f9e136 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/a6f9e136/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user holdenkarau@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #809
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/809/display/redirect?page=changes>
Changes:
[kedin] [Nexmark][SQL] Implement sql query 2
[kenn] Add unshaded configurations in the default profile
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a7f4e56cdd47fd7432cd70e9897fdc18d40eb74c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a7f4e56cdd47fd7432cd70e9897fdc18d40eb74c
Commit message: "Merge pull request #4428: Add unshaded configurations in the default profile"
> git rev-list 0d918b7cab8c4ccb2b5e050501327912161d40a7 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7178035914864911081.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5939822587783960883.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1376013410923103161.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1192285440231473773.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5066611413059226217.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3557364494282888082.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins663497450698164196.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-19 12:00:33,368 edc98009 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/edc98009/pkb.log
2018-01-19 12:00:33,368 edc98009 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-19 12:00:33,369 edc98009 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-19 12:00:33,640 edc98009 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-19 12:00:33,641 edc98009 MainThread INFO Initializing the edw service decoder
2018-01-19 12:00:33,794 edc98009 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-19 12:00:33,797 edc98009 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-19 12:00:33,798 edc98009 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-19 12:00:39,799 edc98009 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-19 12:07:41,823 edc98009 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-19 12:07:41,838 edc98009 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/edc98009/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-19 12:17:41,844 edc98009 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/edc98009/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-19 12:17:41,849 edc98009 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/edc98009/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/edc98009/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/edc98009/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/edc98009/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-19 12:17:41,850 edc98009 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-19 12:17:41,851 edc98009 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-19 12:17:41,853 edc98009 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-19 12:17:41,878 edc98009 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-19 12:17:41,879 edc98009 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/edc98009/pkb.log
2018-01-19 12:17:41,879 edc98009 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/edc98009/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #808
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/808/display/redirect?page=changes>
Changes:
[ankurgoenka] Fixing Work Progress reporting
[chamikara] [BEAM-3496] Add missing dependencies in hadoopInputFormats build.gradle
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 0d918b7cab8c4ccb2b5e050501327912161d40a7 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0d918b7cab8c4ccb2b5e050501327912161d40a7
Commit message: "[BEAM-3496] Add missing dependencies in hadoopInputFormats build.gradle (#4442)"
> git rev-list ca63b05b61b05fd887caf7a2225f6b0c4d55b9b6 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1348652980075422109.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3443535580701281950.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6874526272264012427.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2701832224434989618.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3563668436338927181.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8032337438994077665.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6031072912290678328.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-19 06:00:42,272 1d3b7833 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/1d3b7833/pkb.log
2018-01-19 06:00:42,272 1d3b7833 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-19 06:00:42,273 1d3b7833 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-19 06:00:42,649 1d3b7833 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-19 06:00:42,650 1d3b7833 MainThread INFO Initializing the edw service decoder
2018-01-19 06:00:42,808 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-19 06:00:42,811 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-19 06:00:42,811 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-19 06:00:49,607 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-19 06:08:44,623 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-19 06:08:44,638 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1d3b7833/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-19 06:18:44,644 1d3b7833 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1d3b7833/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-19 06:18:44,649 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1d3b7833/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/1d3b7833/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/1d3b7833/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/1d3b7833/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-19 06:18:44,650 1d3b7833 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-19 06:18:44,651 1d3b7833 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-19 06:18:44,652 1d3b7833 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-19 06:18:44,666 1d3b7833 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-19 06:18:44,667 1d3b7833 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/1d3b7833/pkb.log
2018-01-19 06:18:44,667 1d3b7833 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/1d3b7833/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #807
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/807/display/redirect?page=changes>
Changes:
[samuelw] Update NexmarkLauncher to not cancel jobs that have completed their
[lcwik] [BEAM-3461] Drop redundant beam-YYY-parent and use directory names for
[sidhom] Identify unconsumed reads by URN
[sidhom] Use portable constructs in Flink streaming translator
[chamikara] [BEAM-3217] add HadoopInputFormatIO integration test using DBInputFormat
[lcwik] [BEAM-2863] Migrate away from deprecated methods.
[tgroh] Give UnknownPTransformRunnerFactory a real name
[robertwb] [BEAM-2732] Starting refactor of state tracking in Python (#4375)
[lcwik] Add a more complete FnHarness#main method
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ca63b05b61b05fd887caf7a2225f6b0c4d55b9b6 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ca63b05b61b05fd887caf7a2225f6b0c4d55b9b6
Commit message: "Merge pull request #4435"
> git rev-list ed68f23eba0d8501dab04f086a38133fad1f3cd8 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8939750627309705745.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins382842614401598231.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2628233751867287049.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1071021091926096280.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5171342187378533714.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2330447658726346451.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6745500995295116369.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/1057e5e4'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user samuelw@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #806
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/806/display/redirect?page=changes>
Changes:
[tgroh] Update Dataflow Container Version
[tgroh] Revert "Temporarily disable permared Dataflow integration tests"
[robertwb] Fix reshuffle test on non-direct runners.
[robertwb] Enable counters from sources on workers.
[robertwb] lint
[lcwik] [BEAM-2863] Fix translation of side inputs due to missing method.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ed68f23eba0d8501dab04f086a38133fad1f3cd8 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ed68f23eba0d8501dab04f086a38133fad1f3cd8
Commit message: "Merge pull request #4438 from robertwb/source-counters"
> git rev-list 2f235dd58acce27f713c7072d62cd3a72e3413a1 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6297747686852086882.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4677505034936749218.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6422787165531326409.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8982559036376657162.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3537104194524707240.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4701659925074345903.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1725875462497667847.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/4b10dfdb'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #805
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/805/display/redirect?page=changes>
Changes:
[chamikara] [BEAM-3456] Enable jenkins and large scale scenario in JDBC (#4392)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 2f235dd58acce27f713c7072d62cd3a72e3413a1 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 2f235dd58acce27f713c7072d62cd3a72e3413a1
Commit message: "[BEAM-3456] Enable jenkins and large scale scenario in JDBC (#4392)"
> git rev-list b53100b335eefcb27ec9d0c124fafdda9fda11a4 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2616941644853244786.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins802623999779199375.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins302807635692927063.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2968627028354348190.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4804138810718655643.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6079278858530108224.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
no previously-included directories found matching 'doc/.build'
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2644801377514681959.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-18 12:00:31,535 6cbe0229 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6cbe0229/pkb.log
2018-01-18 12:00:31,535 6cbe0229 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-18 12:00:31,538 6cbe0229 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-18 12:00:31,994 6cbe0229 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-18 12:00:31,995 6cbe0229 MainThread INFO Initializing the edw service decoder
2018-01-18 12:00:32,287 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-18 12:00:32,291 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-18 12:00:32,292 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-18 12:00:47,589 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-18 12:12:10,163 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-18 12:12:10,313 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6cbe0229/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-18 12:22:10,320 6cbe0229 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6cbe0229/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-18 12:22:10,333 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6cbe0229/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6cbe0229/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/6cbe0229/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6cbe0229/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-18 12:22:10,334 6cbe0229 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-18 12:22:10,335 6cbe0229 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-18 12:22:10,337 6cbe0229 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-18 12:22:10,381 6cbe0229 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-18 12:22:10,381 6cbe0229 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6cbe0229/pkb.log
2018-01-18 12:22:10,382 6cbe0229 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6cbe0229/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #804
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/804/display/redirect?page=changes>
Changes:
[ekirpichov] Removes maxBundleSize option accidentally introduced in unrelated PR
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b53100b335eefcb27ec9d0c124fafdda9fda11a4 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b53100b335eefcb27ec9d0c124fafdda9fda11a4
Commit message: "Merge pull request #4432 from jkff/rm-max-bundle-size"
> git rev-list 40c9081d4152462630558b12faa39d62a640d030 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1054261046170647193.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8754832115759458899.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2497388286970322645.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4970884326582752251.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8312985358667767324.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins853833882815504704.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6131593532790405545.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-18 06:00:33,857 6300f5d2 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/6300f5d2/pkb.log
2018-01-18 06:00:33,857 6300f5d2 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-18 06:00:33,858 6300f5d2 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-18 06:00:34,296 6300f5d2 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-18 06:00:34,296 6300f5d2 MainThread INFO Initializing the edw service decoder
2018-01-18 06:00:34,435 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-18 06:00:34,438 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-18 06:00:34,439 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-18 06:00:40,920 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-18 06:07:31,020 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-18 06:07:31,038 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6300f5d2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-18 06:17:31,045 6300f5d2 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6300f5d2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-18 06:17:31,053 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/6300f5d2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6300f5d2/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/6300f5d2/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/6300f5d2/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-18 06:17:31,054 6300f5d2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-18 06:17:31,055 6300f5d2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-18 06:17:31,057 6300f5d2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-18 06:17:31,122 6300f5d2 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-18 06:17:31,123 6300f5d2 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/6300f5d2/pkb.log
2018-01-18 06:17:31,123 6300f5d2 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/6300f5d2/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user ekirpichov@gmail.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #803
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/803/display/redirect?page=changes>
Changes:
[niemo] Update BigTableSource to use multiple key ranges
[niemo] Add tests for supporting multiple key ranges.
[niemo] Use multiple key ranges in BigtableServiceImpl
[tgroh] Add wire_coder_id to RemoteGrpcPort
[tgroh] Add joda-time dependency to java/fn-execution
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 40c9081d4152462630558b12faa39d62a640d030 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 40c9081d4152462630558b12faa39d62a640d030
Commit message: "Merge pull request #4386"
> git rev-list 4da548ec2191c25c4524b5a1076e09f18aeb4446 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1751550996273696613.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2163746126221078473.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins735067987687937574.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3065146642491944025.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7545760625304875967.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2624357635865480319.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1904375457931579512.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/cb28f5bf'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user niemo@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Build failed in Jenkins: beam_PerformanceTests_Python #802
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/802/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 4da548ec2191c25c4524b5a1076e09f18aeb4446 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 4da548ec2191c25c4524b5a1076e09f18aeb4446
Commit message: "Merge pull request #4224"
> git rev-list 4da548ec2191c25c4524b5a1076e09f18aeb4446 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4714855682459180193.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5948525854507735433.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins213902006961179364.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5621201568096929840.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins844557816106572186.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied (use --upgrade to upgrade): colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1330450193949001669.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Running setup.py (path:<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py)> egg_info for package from file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-apitools>=0.5.18,<=0.5.20 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): googleapis-common-protos>=1.5.2,<2.0dev in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
Running setup.py develop for apache-beam
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/setup.py>:66: UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is recommended.
_PIP_VERSION, REQUIRED_PIP_VERSION
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
Creating <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link> (link to .)
Adding apache-beam 2.3.0.dev to easy-install.pth file
Installed <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4784006087378422877.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/lib/python2.7/os.py",> line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/f8997c67'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #801
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/801/display/redirect?page=changes>
Changes:
[kedin] [Nexmark][SQL] Implement Query1
[tgroh] Move BeamFnDataGrpcMultiplexer to java-fn-execution
[tgroh] Mark BeamFnDataGrpcMultiplexer ApiServiceDescriptor as Nullable
[tgroh] Enable Java8 for fn execution modules
[tgroh] Update GRPC Multiplexer and Associated to Java8
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 4da548ec2191c25c4524b5a1076e09f18aeb4446 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 4da548ec2191c25c4524b5a1076e09f18aeb4446
Commit message: "Merge pull request #4224"
> git rev-list e977cdff1718b2316115f63903762380c15530de # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1137789986977891174.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7159714549836575533.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6958673806892171360.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins937354010526847486.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6423037473912671204.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins477008895984791923.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7597089381591433151.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-17 12:00:39,410 9a3e4af0 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/9a3e4af0/pkb.log
2018-01-17 12:00:39,411 9a3e4af0 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-17 12:00:39,411 9a3e4af0 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-17 12:00:39,798 9a3e4af0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-17 12:00:39,799 9a3e4af0 MainThread INFO Initializing the edw service decoder
2018-01-17 12:00:39,985 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-17 12:00:39,989 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-17 12:00:39,990 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-17 12:00:49,298 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-17 12:11:45,602 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-17 12:11:45,748 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9a3e4af0/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-17 12:21:45,756 9a3e4af0 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9a3e4af0/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-17 12:21:45,767 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9a3e4af0/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9a3e4af0/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/9a3e4af0/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9a3e4af0/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-17 12:21:45,768 9a3e4af0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-17 12:21:45,769 9a3e4af0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-17 12:21:45,772 9a3e4af0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-17 12:21:45,819 9a3e4af0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-17 12:21:45,820 9a3e4af0 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/9a3e4af0/pkb.log
2018-01-17 12:21:45,820 9a3e4af0 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/9a3e4af0/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user kedin@google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #800
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/800/display/redirect?page=changes>
Changes:
[klk] Move DoFnAndMainOutput to top level
[klk] Move DoFnSignature helpers to DoFnSignatures and rename
[klk] Clarify names and visibility of ParDoTranslation helpers
[klk] Make default ParDo output tag explicit
[klk] Fix coders in Java SDK harness
[klk] More logging and error checking in ProcessBundleHandler
[klk] Pass portable transform IDs in Dataflow runner
[klk] Correct getAdditionalInputs for Dataflow single-output ParDo override
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e977cdff1718b2316115f63903762380c15530de (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f e977cdff1718b2316115f63903762380c15530de
Commit message: "Merge pull request #4356: [BEAM-2963] Pass portable transform IDs in Dataflow translation"
> git rev-list 70a520ce9c0930742cf44efad6dbad1a5df29734 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1058262209136290439.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins986829844133369639.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1665383366362244846.sh
+ virtualenv .env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3461485980639252014.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1016776078460366873.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5370589051890371600.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Not uninstalling apache-beam at <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python,> outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2873077102453304481.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-17 06:00:40,568 1076fc2c MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/1076fc2c/pkb.log
2018-01-17 06:00:40,569 1076fc2c MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-17 06:00:40,571 1076fc2c MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-17 06:00:40,908 1076fc2c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-17 06:00:40,909 1076fc2c MainThread INFO Initializing the edw service decoder
2018-01-17 06:00:41,067 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-17 06:00:41,071 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-17 06:00:41,071 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-17 06:00:47,483 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-17 06:10:14,955 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-17 06:10:15,011 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1076fc2c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-17 06:20:15,052 1076fc2c Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1076fc2c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-17 06:20:15,062 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/1076fc2c/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/1076fc2c/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/1076fc2c/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/1076fc2c/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-17 06:20:15,063 1076fc2c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-17 06:20:15,159 1076fc2c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-17 06:20:15,162 1076fc2c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-17 06:20:15,193 1076fc2c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-17 06:20:15,194 1076fc2c MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/1076fc2c/pkb.log
2018-01-17 06:20:15,194 1076fc2c MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/1076fc2c/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #799
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/799/display/redirect?page=changes>
Changes:
[rmannibucau] better error message when there are conflicting anonymous names
[rmannibucau] adding comments to make the code more obvious and extracting the
[rmannibucau] adding tostring in ParDo.*Output and removing unstableNames variable
[daniel.o.programmer] [BEAM-3126] Creating flatten operation for bundle processor.
[rmannibucau] adressing comments
[dariusz.aniszewski] use VirtualEnv for PerfKit jobs
[lcwik] [BEAM-3125] Creating flatten runner, unfinished.
[lcwik] [BEAM-3125] Creating flatten runner
[lcwik] [BEAM-3125] Fixing little nitpicks and typos.
[lcwik] [BEAM-3126] Fixes for flatten runner.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 70a520ce9c0930742cf44efad6dbad1a5df29734 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 70a520ce9c0930742cf44efad6dbad1a5df29734
Commit message: "Merge pull request #4408 from youngoli/beam3126"
> git rev-list da5020081bd474188ced5a00d064b2e0e12e1c86 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1741344967073341215.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8012169674387820574.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6743291882370260421.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2090415624259403056.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3528517846872795090.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/8d123865'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user daniel.o.programmer@gmail.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #798
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/798/display/redirect?page=changes>
Changes:
[jbonofre] [BEAM-3438] Disable it on Kinesis
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision da5020081bd474188ced5a00d064b2e0e12e1c86 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f da5020081bd474188ced5a00d064b2e0e12e1c86
Commit message: "Merge pull request #4379 from jbonofre/BEAM-3438"
> git rev-list d11b9e9560131f55b418a13a7d10401c2135fb33 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8328157389362246067.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8173405402697879867.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2458542523866808290.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4193849295238578571.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3248238501669904248.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-16 18:00:32,336 a93fe9a1 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/a93fe9a1/pkb.log
2018-01-16 18:00:32,337 a93fe9a1 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-16 18:00:32,337 a93fe9a1 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-16 18:00:32,893 a93fe9a1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-16 18:00:32,894 a93fe9a1 MainThread INFO Initializing the edw service decoder
2018-01-16 18:00:33,029 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-16 18:00:33,033 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-16 18:00:33,034 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-16 18:00:38,461 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-16 18:08:48,945 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-16 18:08:48,997 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a93fe9a1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-16 18:18:49,003 a93fe9a1 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a93fe9a1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-16 18:18:49,008 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/a93fe9a1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a93fe9a1/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/a93fe9a1/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/a93fe9a1/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-16 18:18:49,008 a93fe9a1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-16 18:18:49,009 a93fe9a1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-16 18:18:49,011 a93fe9a1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-16 18:18:49,034 a93fe9a1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-16 18:18:49,034 a93fe9a1 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/a93fe9a1/pkb.log
2018-01-16 18:18:49,034 a93fe9a1 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/a93fe9a1/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #797
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/797/display/redirect?page=changes>
Changes:
[thw] [BEAM-3240] Fix quickstart archetypes to include expected Hadoop
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision d11b9e9560131f55b418a13a7d10401c2135fb33 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f d11b9e9560131f55b418a13a7d10401c2135fb33
Commit message: "Merge pull request #4402: [BEAM-3240] Fix quickstart archetypes to include expected Hadoop dependencies under Apex profile"
> git rev-list de8744429f99f53b08acbf29490c22672b4a6ccc # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4483872829553001342.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3326430452556998922.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3199345707389669759.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7671879809662739132.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2246748955127168991.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-16 12:00:36,788 c18721c1 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/c18721c1/pkb.log
2018-01-16 12:00:36,789 c18721c1 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-16 12:00:36,790 c18721c1 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-16 12:00:37,072 c18721c1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-16 12:00:37,072 c18721c1 MainThread INFO Initializing the edw service decoder
2018-01-16 12:00:37,211 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-16 12:00:37,214 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-16 12:00:37,215 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-16 12:00:43,731 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-16 12:10:21,893 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-16 12:10:21,913 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/c18721c1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-16 12:20:21,921 c18721c1 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/c18721c1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-16 12:20:21,928 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/c18721c1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/c18721c1/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/c18721c1/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/c18721c1/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-16 12:20:21,928 c18721c1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-16 12:20:21,929 c18721c1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-16 12:20:21,931 c18721c1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-16 12:20:21,956 c18721c1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-16 12:20:21,957 c18721c1 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/c18721c1/pkb.log
2018-01-16 12:20:21,958 c18721c1 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/c18721c1/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #796
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/796/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision de8744429f99f53b08acbf29490c22672b4a6ccc (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f de8744429f99f53b08acbf29490c22672b4a6ccc
Commit message: "Merge pull request #4415 from RyanSkraba/BEAM-3454-stop-unbounded-srcs"
> git rev-list de8744429f99f53b08acbf29490c22672b4a6ccc # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6803388084946768577.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2448803547349988035.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8638422547808862397.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4827523656790528903.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3474705113231314801.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-16 06:00:19,814 374e6bb1 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/374e6bb1/pkb.log
2018-01-16 06:00:19,814 374e6bb1 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-16 06:00:19,815 374e6bb1 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-16 06:00:20,100 374e6bb1 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-16 06:00:20,101 374e6bb1 MainThread INFO Initializing the edw service decoder
2018-01-16 06:00:20,245 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-16 06:00:20,248 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-16 06:00:20,248 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-16 06:00:26,238 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-16 06:07:07,381 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-16 06:07:07,400 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/374e6bb1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-16 06:17:07,407 374e6bb1 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/374e6bb1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-16 06:17:07,416 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/374e6bb1/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/374e6bb1/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/374e6bb1/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/374e6bb1/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-16 06:17:07,417 374e6bb1 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-16 06:17:07,418 374e6bb1 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-16 06:17:07,421 374e6bb1 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-16 06:17:07,443 374e6bb1 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-16 06:17:07,444 374e6bb1 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/374e6bb1/pkb.log
2018-01-16 06:17:07,445 374e6bb1 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/374e6bb1/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #795
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/795/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision de8744429f99f53b08acbf29490c22672b4a6ccc (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f de8744429f99f53b08acbf29490c22672b4a6ccc
Commit message: "Merge pull request #4415 from RyanSkraba/BEAM-3454-stop-unbounded-srcs"
> git rev-list de8744429f99f53b08acbf29490c22672b4a6ccc # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8301593876143000988.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8965731625632813967.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8979518562253458452.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2875343219387859500.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3716001525848832155.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-16 00:00:30,440 ea167e76 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/ea167e76/pkb.log
2018-01-16 00:00:30,441 ea167e76 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-16 00:00:30,442 ea167e76 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-16 00:00:30,836 ea167e76 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-16 00:00:30,836 ea167e76 MainThread INFO Initializing the edw service decoder
2018-01-16 00:00:30,985 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-16 00:00:30,988 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-16 00:00:30,989 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-16 00:00:38,879 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-16 00:09:07,394 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-16 00:09:07,412 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ea167e76/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-16 00:19:07,420 ea167e76 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ea167e76/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-16 00:19:07,427 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ea167e76/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ea167e76/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/ea167e76/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ea167e76/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-16 00:19:07,428 ea167e76 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-16 00:19:07,429 ea167e76 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-16 00:19:07,432 ea167e76 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-16 00:19:07,484 ea167e76 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-16 00:19:07,485 ea167e76 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ea167e76/pkb.log
2018-01-16 00:19:07,485 ea167e76 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ea167e76/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #794
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/794/display/redirect?page=changes>
Changes:
[iemejia] Add missing protobuf-java dependency on runners/java-fn-execution module
[ryan] [BEAM-3454] Use multiple stopping conditions on JmsIO, AmqpIO, MqttIO.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision de8744429f99f53b08acbf29490c22672b4a6ccc (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f de8744429f99f53b08acbf29490c22672b4a6ccc
Commit message: "Merge pull request #4415 from RyanSkraba/BEAM-3454-stop-unbounded-srcs"
> git rev-list dc40445ca9bb86f9f046912e2fbb944610297ce4 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9053323229635577109.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8774788961801622418.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8048055137951976982.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9208683718584985641.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5864263644741675089.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-15 18:05:20,254 02d086d7 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/02d086d7/pkb.log
2018-01-15 18:05:20,254 02d086d7 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-15 18:05:20,255 02d086d7 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-15 18:05:20,439 02d086d7 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-15 18:05:20,439 02d086d7 MainThread INFO Initializing the edw service decoder
2018-01-15 18:05:20,562 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-15 18:05:20,565 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-15 18:05:20,565 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-15 18:05:26,836 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-15 18:12:35,092 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-15 18:12:35,107 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/02d086d7/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-15 18:22:35,113 02d086d7 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/02d086d7/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-15 18:22:35,122 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/02d086d7/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/02d086d7/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/02d086d7/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/02d086d7/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-15 18:22:35,123 02d086d7 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-15 18:22:35,124 02d086d7 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-15 18:22:35,125 02d086d7 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-15 18:22:35,181 02d086d7 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-15 18:22:35,182 02d086d7 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/02d086d7/pkb.log
2018-01-15 18:22:35,182 02d086d7 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/02d086d7/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #793
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/793/display/redirect>
------------------------------------------
[...truncated 746.88 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/7167f079/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/7167f079/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/7167f079/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/7167f079/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/7167f079/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 9.089 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 4.101 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.135 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 8.744 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.463 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 4.698 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.177 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 13.701 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.137 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 23.275 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.095 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.007 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.602 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.624 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.666 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.068 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 3.613 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.100 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 4.116 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.941 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.979 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.714 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 3.207 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 4.366 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.053 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.802 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.916 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 6.870 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 9.387 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.361 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 6.945 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.400 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.856 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.243 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 22.380 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 7.717 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 11.730 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.852 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.796 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 3.091 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 3.485 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.991 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.880 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.741 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 6.560 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 6.417 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.790 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.058 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 9.013 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.075 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.463 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 23.029 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.803 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 26.417 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.290 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.325 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 3.322 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.720 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 21.219 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.129 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.800 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 9.837 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 3.902 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.701 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 17.304 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 3.856 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 3.071 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:34 min
[INFO] Finished at: 2018-01-15T12:07:17Z
[INFO] Final Memory: 344M/1910M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ find /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.24 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-15 12:07:18,874 7167f079 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-15 12:07:18,891 7167f079 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-15 12:17:18,898 7167f079 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-15 12:17:18,911 7167f079 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/7167f079/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-15 12:17:18,911 7167f079 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-15 12:17:18,912 7167f079 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-15 12:17:18,914 7167f079 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-15 12:17:18,941 7167f079 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-15 12:17:18,943 7167f079 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/7167f079/pkb.log
2018-01-15 12:17:18,943 7167f079 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/7167f079/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #792
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/792/display/redirect>
------------------------------------------
[...truncated 746.51 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 3.198 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 2.912 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.063 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 9.722 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.915 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 4.916 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.129 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 8.265 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.100 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 19.625 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.073 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.710 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.006 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 3.998 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.073 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.055 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.742 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.009 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.960 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.385 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.505 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.613 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 2.044 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 2.058 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.046 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.695 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.529 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.026 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.418 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.150 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.776 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 3.616 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.936 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.093 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 12.783 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 5.830 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 8.377 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.381 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.480 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.608 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 2.694 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.363 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.371 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.272 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.415 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.931 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.816 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.024 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 7.450 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.032 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.404 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 21.175 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.695 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 25.234 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.101 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.022 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.621 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.499 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 16.854 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 2.762 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 14.291 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 9.599 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 4.081 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.661 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 13.876 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 3.842 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 2.867 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:21 min
[INFO] Finished at: 2018-01-15T06:05:49Z
[INFO] Final Memory: 349M/2143M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 3.98 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-15 06:05:50,775 ec2bdb55 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-15 06:05:50,792 ec2bdb55 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-15 06:15:50,799 ec2bdb55 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-15 06:15:50,810 ec2bdb55 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/ec2bdb55/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-15 06:15:50,810 ec2bdb55 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-15 06:15:50,811 ec2bdb55 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-15 06:15:50,813 ec2bdb55 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-15 06:15:50,827 ec2bdb55 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-15 06:15:50,828 ec2bdb55 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/ec2bdb55/pkb.log
2018-01-15 06:15:50,828 ec2bdb55 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/ec2bdb55/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #791
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/791/display/redirect>
------------------------------------------
[...truncated 748.59 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/72434fe8/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/72434fe8/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/72434fe8/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/72434fe8/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/72434fe8/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 2.959 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 2.272 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.034 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 7.637 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 5.285 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 6.231 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.168 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 8.747 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.119 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 21.495 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.081 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 1.991 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.668 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.340 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.573 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.055 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.987 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.060 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 3.021 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.505 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.585 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.810 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.848 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 1.993 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.033 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.627 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.722 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 11.355 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 9.131 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.329 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 4.121 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.145 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.929 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.373 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 13.184 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 5.850 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 8.933 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.520 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.522 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.890 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 2.908 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.786 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.526 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.432 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.410 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.897 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.808 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.020 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 7.919 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.032 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.470 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 22.115 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.957 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 27.226 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.053 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.118 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.556 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.438 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 18.242 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.092 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 14.888 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 9.577 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 3.880 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.602 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 19.301 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 3.760 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 2.978 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:47 min
[INFO] Finished at: 2018-01-15T00:06:13Z
[INFO] Final Memory: 351M/2123M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ find /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.11 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-15 00:06:15,108 72434fe8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-15 00:06:15,126 72434fe8 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-15 00:16:15,134 72434fe8 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-15 00:16:15,145 72434fe8 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/72434fe8/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-15 00:16:15,147 72434fe8 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-15 00:16:15,148 72434fe8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-15 00:16:15,152 72434fe8 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-15 00:16:15,201 72434fe8 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-15 00:16:15,202 72434fe8 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/72434fe8/pkb.log
2018-01-15 00:16:15,202 72434fe8 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/72434fe8/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #790
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/790/display/redirect>
------------------------------------------
[...truncated 746.51 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 8.954 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 4.224 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.071 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 9.139 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.363 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 4.930 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.095 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 13.834 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.124 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 22.357 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.082 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.238 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.415 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.773 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.930 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.072 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 3.587 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.164 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 4.098 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 3.068 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.835 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.617 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 3.694 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 3.880 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.048 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.961 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.569 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 6.326 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 9.194 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.254 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 6.890 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.757 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.796 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.124 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 20.790 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 7.376 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 11.426 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.848 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.608 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.975 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 3.366 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.910 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.658 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.509 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 6.200 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 6.225 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.661 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.035 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 8.963 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.056 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.493 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 22.939 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.827 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 26.548 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.284 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.233 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 3.149 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.760 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 21.046 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.077 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.189 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 9.694 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 3.900 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.739 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 16.106 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 3.665 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 2.896 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:26 min
[INFO] Finished at: 2018-01-14T18:07:03Z
[INFO] Final Memory: 345M/1492M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.16 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-14 18:07:05,383 7f9ddaca MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-14 18:07:05,399 7f9ddaca MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-14 18:17:05,407 7f9ddaca Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-14 18:17:05,417 7f9ddaca MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/7f9ddaca/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-14 18:17:05,418 7f9ddaca MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-14 18:17:05,419 7f9ddaca MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-14 18:17:05,421 7f9ddaca MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-14 18:17:05,455 7f9ddaca MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-14 18:17:05,456 7f9ddaca MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/7f9ddaca/pkb.log
2018-01-14 18:17:05,456 7f9ddaca MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/7f9ddaca/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #789
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/789/display/redirect>
------------------------------------------
[...truncated 746.42 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/5435d0fc/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/5435d0fc/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/5435d0fc/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/5435d0fc/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/5435d0fc/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 2.748 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 2.213 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.035 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 6.631 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 3.767 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 4.598 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.120 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 10.781 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.108 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 19.601 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.085 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.186 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.542 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.274 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.801 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.054 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.749 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.188 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 2.967 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.458 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.502 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.592 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.863 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 1.861 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.039 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.520 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.521 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.521 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.999 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.230 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 4.194 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.297 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.881 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.275 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 13.671 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.523 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 9.202 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.384 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.624 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.869 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 2.789 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.556 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.693 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.371 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.626 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 4.093 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.775 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.021 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 7.851 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.031 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.296 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 22.794 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.827 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 27.446 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.077 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.135 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.570 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.534 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 18.368 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.202 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.326 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 10.005 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 3.914 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.654 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 14.053 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 3.831 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 2.883 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:34 min
[INFO] Finished at: 2018-01-14T12:06:29Z
[INFO] Final Memory: 348M/2040M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.00 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-14 12:06:30,405 5435d0fc MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-14 12:06:30,421 5435d0fc MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-14 12:16:30,428 5435d0fc Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-14 12:16:30,442 5435d0fc MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/5435d0fc/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-14 12:16:30,443 5435d0fc MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-14 12:16:30,444 5435d0fc MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-14 12:16:30,446 5435d0fc MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-14 12:16:30,471 5435d0fc MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-14 12:16:30,473 5435d0fc MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/5435d0fc/pkb.log
2018-01-14 12:16:30,473 5435d0fc MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/5435d0fc/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #788
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/788/display/redirect>
------------------------------------------
[...truncated 746.50 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/750d72cb/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/750d72cb/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/750d72cb/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/750d72cb/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/750d72cb/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 4.600 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 2.864 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.064 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 8.935 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.730 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 5.125 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.119 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 8.694 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.084 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 20.170 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.056 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.149 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.387 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.447 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.966 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.063 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 3.528 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.112 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 3.202 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.698 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.713 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.656 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 2.143 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 2.253 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.044 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.827 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.629 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.543 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.866 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.515 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.965 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.072 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.797 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.289 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 13.766 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.602 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 9.502 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.317 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.845 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.933 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 3.297 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.815 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.615 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.804 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.755 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.969 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.689 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.020 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 8.315 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.032 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.742 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 22.298 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 4.164 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 27.050 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.249 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.395 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.655 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.649 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 19.680 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.286 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.479 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 10.383 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 3.921 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.779 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 15.401 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 4.022 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 3.323 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:48 min
[INFO] Finished at: 2018-01-14T06:06:15Z
[INFO] Final Memory: 349M/2006M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.40 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-14 06:06:17,471 750d72cb MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-14 06:06:17,487 750d72cb MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-14 06:16:17,493 750d72cb Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-14 06:16:17,505 750d72cb MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/750d72cb/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-14 06:16:17,506 750d72cb MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-14 06:16:17,507 750d72cb MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-14 06:16:17,509 750d72cb MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-14 06:16:17,541 750d72cb MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-14 06:16:17,542 750d72cb MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/750d72cb/pkb.log
2018-01-14 06:16:17,542 750d72cb MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/750d72cb/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #787
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/787/display/redirect?page=changes>
Changes:
[iemejia] [BEAM-3467] Remove Java 7 from the docker development images
[klk] Temporarily disable permared Dataflow integration tests
[iemejia] Fix unused dependency error on Flink runner
------------------------------------------
[...truncated 748.55 KB...]
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/704c5099/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/704c5099/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/704c5099/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/704c5099/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/704c5099/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 10.395 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 5.508 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.112 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 11.168 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.536 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 4.862 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.176 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 15.006 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.160 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 24.200 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.098 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.585 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 5.017 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 5.216 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 14.375 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.099 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 4.634 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.483 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 4.645 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 3.339 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 1.184 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.737 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 4.157 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 4.372 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.065 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 5.051 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 3.154 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 8.487 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 10.608 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.874 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 7.595 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 5.056 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 2.070 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 4.001 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 23.407 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 8.584 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 14.074 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.951 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 3.190 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 3.266 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 3.926 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 3.244 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 3.002 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.697 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 6.487 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 7.321 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.992 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.081 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 10.123 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.059 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 4.135 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 25.522 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.974 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 27.029 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.372 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.379 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 3.755 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 3.232 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 23.551 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.514 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.606 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 11.432 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 4.674 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.793 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 18.802 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 4.889 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 3.191 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:17 min
[INFO] Finished at: 2018-01-14T00:08:03Z
[INFO] Final Memory: 350M/2146M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.80 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-14 00:08:04,932 704c5099 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-14 00:08:04,948 704c5099 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-14 00:18:04,955 704c5099 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-14 00:18:04,964 704c5099 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/704c5099/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-14 00:18:04,965 704c5099 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-14 00:18:04,966 704c5099 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-14 00:18:04,970 704c5099 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-14 00:18:04,982 704c5099 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-14 00:18:04,983 704c5099 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/704c5099/pkb.log
2018-01-14 00:18:04,983 704c5099 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/704c5099/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user klk@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #786
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/786/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-3478] Fix checkstyle issue in runners/flink
------------------------------------------
[...truncated 746.60 KB...]
[INFO] Excluding io.grpc:grpc-protobuf-lite:jar:1.2.0 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/f3780ea2/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/f3780ea2/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/f3780ea2/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/f3780ea2/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/f3780ea2/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 5.544 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 3.779 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.039 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 9.690 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 5.525 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 5.093 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.178 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 8.316 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.128 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 22.296 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.074 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.116 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.383 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.785 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.954 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.061 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 3.370 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.332 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 3.128 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.405 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.574 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.799 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 1.873 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 2.278 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.044 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.875 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.584 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.171 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 8.846 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.600 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 4.049 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.313 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 1.757 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.409 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 13.511 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 6.245 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 10.271 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.528 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.686 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.846 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 2.881 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.548 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.520 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.354 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.272 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 3.843 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.659 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.022 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 7.978 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.040 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.568 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 21.177 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 3.879 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 26.119 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.173 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.144 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.614 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.540 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 19.031 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.397 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 14.608 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 9.870 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 3.999 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.711 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 16.742 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 3.661 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 3.105 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:47 min
[INFO] Finished at: 2018-01-13T18:06:20Z
[INFO] Final Memory: 347M/1985M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ find /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ find /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.18 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-13 18:06:22,311 f3780ea2 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-13 18:06:22,328 f3780ea2 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-13 18:16:22,335 f3780ea2 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-13 18:16:22,347 f3780ea2 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/f3780ea2/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-13 18:16:22,348 f3780ea2 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-13 18:16:22,349 f3780ea2 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-13 18:16:22,351 f3780ea2 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-13 18:16:22,404 f3780ea2 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-13 18:16:22,404 f3780ea2 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/f3780ea2/pkb.log
2018-01-13 18:16:22,407 f3780ea2 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/f3780ea2/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #785
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/785/display/redirect?page=changes>
Changes:
[rmannibucau] BEAM-682 try to define generated classes in the correct classloader
[github] [BEAM-3336] Fix thread safety issues of MqttIOTest (#4406)
------------------------------------------
[...truncated 746.39 KB...]
[INFO] Excluding io.grpc:grpc-protobuf-lite:jar:1.2.0 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/56ab5f81/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/56ab5f81/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/56ab5f81/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/56ab5f81/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/56ab5f81/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 3.328 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 2.072 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.036 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 7.067 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 4.186 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 5.544 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.131 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 9.743 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.119 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 22.149 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.081 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.264 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.925 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.568 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 11.324 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.078 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 2.945 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.099 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 3.058 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 2.475 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.715 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.635 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 2.011 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 2.018 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.049 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 3.347 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.693 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 5.033 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 9.001 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.481 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 3.940 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.012 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 2.090 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.393 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 13.642 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 5.843 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 9.268 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 3.454 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 2.690 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 2.930 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 3.037 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 2.894 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 2.709 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 2.526 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 4.493 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 4.095 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.601 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.022 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 7.872 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.033 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.384 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 21.844 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 4.083 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 26.610 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.211 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.144 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 2.715 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 2.599 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 17.593 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.187 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 15.056 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 9.894 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 4.146 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.647 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 14.469 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 4.094 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 3.138 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:37 min
[INFO] Finished at: 2018-01-13T12:06:07Z
[INFO] Final Memory: 343M/1914M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ find /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 4.10 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-13 12:06:08,616 56ab5f81 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-13 12:06:08,634 56ab5f81 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-13 12:16:08,640 56ab5f81 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-13 12:16:08,650 56ab5f81 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/56ab5f81/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-13 12:16:08,651 56ab5f81 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-13 12:16:08,653 56ab5f81 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-13 12:16:08,656 56ab5f81 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-13 12:16:08,709 56ab5f81 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-13 12:16:08,710 56ab5f81 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/56ab5f81/pkb.log
2018-01-13 12:16:08,710 56ab5f81 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/56ab5f81/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #784
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/784/display/redirect?page=changes>
Changes:
[github] [BEAM-3436] Fix RetryHttpRequestInitializerTest to use fake
------------------------------------------
[...truncated 748.50 KB...]
[INFO] Excluding io.grpc:grpc-protobuf-lite:jar:1.2.0 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-fn-execution:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-common:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-buffer:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-transport:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-resolver:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.4 from the shaded jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.1 from the shaded jar.
[INFO] Excluding org.apache.beam:beam-sdks-java-core:jar:2.3.0-SNAPSHOT from the shaded jar.
[INFO] Excluding com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.8.9 from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.8.9 from the shaded jar.
[INFO] Excluding net.bytebuddy:byte-buddy:jar:1.6.8 from the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.8.2 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 from the shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.7 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.5 from the shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.4 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.14 from the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.6 from the shaded jar.
[INFO] Excluding io.grpc:grpc-core:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.errorprone:error_prone_annotations:jar:2.0.15 from the shaded jar.
[INFO] Excluding io.grpc:grpc-context:jar:1.2.0 from the shaded jar.
[INFO] Excluding com.google.instrumentation:instrumentation-api:jar:0.3.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-stub:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.grpc:grpc-netty:jar:1.2.0 from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http2:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-http:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-handler-proxy:jar:4.1.8.Final from the shaded jar.
[INFO] Excluding io.netty:netty-codec-socks:jar:4.1.8.Final from the shaded jar.
[INFO] Including com.google.guava:guava:jar:20.0 in the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.25 from the shaded jar.
[INFO] Excluding com.google.protobuf:protobuf-java-util:jar:3.2.0 from the shaded jar.
[INFO] Excluding com.google.code.gson:gson:jar:2.7 from the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/3b7012e3/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT.jar with /tmp/perfkitbenchmarker/runs/3b7012e3/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded.jar
[INFO] Replacing original test artifact with shaded test artifact.
[INFO] Replacing /tmp/perfkitbenchmarker/runs/3b7012e3/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-tests.jar with /tmp/perfkitbenchmarker/runs/3b7012e3/beam/runners/java-fn-execution/target/beam-runners-java-fn-execution-2.3.0-SNAPSHOT-shaded-tests.jar
[INFO] Dependency-reduced POM written at: /tmp/perfkitbenchmarker/runs/3b7012e3/beam/runners/java-fn-execution/dependency-reduced-pom.xml
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:analyze-only (default) @ beam-runners-java-fn-execution ---
[WARNING] Used undeclared dependencies found:
[WARNING] com.google.protobuf:protobuf-java:jar:3.2.0:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Beam :: Parent .............................. SUCCESS [ 10.641 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SUCCESS [ 4.713 s]
[INFO] Apache Beam :: Model ............................... SUCCESS [ 0.063 s]
[INFO] Apache Beam :: Model :: Pipeline ................... SUCCESS [ 9.457 s]
[INFO] Apache Beam :: Model :: Job Management ............. SUCCESS [ 5.580 s]
[INFO] Apache Beam :: Model :: Fn Execution ............... SUCCESS [ 5.454 s]
[INFO] Apache Beam :: SDKs ................................ SUCCESS [ 0.554 s]
[INFO] Apache Beam :: SDKs :: Go .......................... SUCCESS [ 14.578 s]
[INFO] Apache Beam :: SDKs :: Java ........................ SUCCESS [ 0.123 s]
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SUCCESS [ 26.672 s]
[INFO] Apache Beam :: Runners ............................. SUCCESS [ 0.345 s]
[INFO] Apache Beam :: Runners :: Local Java Core .......... SUCCESS [ 2.397 s]
[INFO] Apache Beam :: Runners :: Core Construction Java ... SUCCESS [ 4.801 s]
[INFO] Apache Beam :: Runners :: Core Java ................ SUCCESS [ 4.604 s]
[INFO] Apache Beam :: Runners :: Direct Java .............. SUCCESS [ 12.569 s]
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SUCCESS [ 0.123 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: AMQP .......... SUCCESS [ 3.944 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Common ........ SUCCESS [ 2.305 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Cassandra ..... SUCCESS [ 4.017 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch . SUCCESS [ 3.145 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests SUCCESS [ 0.903 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: Common SUCCESS [ 0.692 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 2.x SUCCESS [ 4.601 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Elasticsearch-Tests :: 5.x SUCCESS [ 5.076 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SUCCESS [ 0.096 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Google Cloud Platform Core SUCCESS [ 4.591 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Protobuf SUCCESS [ 2.901 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SUCCESS [ 8.426 s]
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SUCCESS [ 9.884 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: File-based-io-tests SUCCESS [ 2.640 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop Common . SUCCESS [ 8.033 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop File System SUCCESS [ 4.900 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop ........ SUCCESS [ 2.007 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: input-format SUCCESS [ 3.500 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Hadoop :: jdk1.8-tests SUCCESS [ 23.593 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HBase ......... SUCCESS [ 8.655 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: HCatalog ...... SUCCESS [ 13.215 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SUCCESS [ 4.197 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SUCCESS [ 3.855 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SUCCESS [ 4.824 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SUCCESS [ 5.581 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SUCCESS [ 4.512 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SUCCESS [ 3.493 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Redis ......... SUCCESS [ 4.403 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Solr .......... SUCCESS [ 9.007 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: Tika .......... SUCCESS [ 7.319 s]
[INFO] Apache Beam :: SDKs :: Java :: IO :: XML ........... SUCCESS [ 2.817 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SUCCESS [ 0.060 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SUCCESS [ 9.711 s]
[INFO] Apache Beam :: Examples ............................ SUCCESS [ 0.168 s]
[INFO] Apache Beam :: Examples :: Java .................... SUCCESS [ 3.475 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SUCCESS [ 24.790 s]
[INFO] Apache Beam :: Examples :: Java 8 .................. SUCCESS [ 4.361 s]
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SUCCESS [ 31.502 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Jackson SUCCESS [ 2.944 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SUCCESS [ 2.877 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sketching SUCCESS [ 3.296 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter SUCCESS [ 3.225 s]
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: SQL ... SUCCESS [ 26.189 s]
[INFO] Apache Beam :: SDKs :: Java :: Fn Execution ........ SUCCESS [ 3.234 s]
[INFO] Apache Beam :: SDKs :: Java :: Nexmark ............. SUCCESS [ 19.718 s]
[INFO] Apache Beam :: SDKs :: Java :: Harness ............. SUCCESS [ 16.784 s]
[INFO] Apache Beam :: SDKs :: Java :: Container ........... SUCCESS [ 4.480 s]
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SUCCESS [ 0.778 s]
[INFO] Apache Beam :: SDKs :: Python ...................... SUCCESS [ 20.821 s]
[INFO] Apache Beam :: SDKs :: Python :: Container ......... SUCCESS [ 8.346 s]
[INFO] Apache Beam :: Runners :: Java Fn Execution ........ FAILURE [ 6.219 s]
[INFO] Apache Beam :: Runners :: Java Local Artifact Service SKIPPED
[INFO] Apache Beam :: Runners :: Reference ................ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Java ........ SKIPPED
[INFO] Apache Beam :: Runners :: Reference :: Job Orchestrator SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform .... SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCE metadata provisioning SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Platform :: GCS artifact proxy SKIPPED
[INFO] Apache Beam :: Runners :: Gearpump ................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Aggregated Javadoc .. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:46 min
[INFO] Finished at: 2018-01-13T06:08:25Z
[INFO] Final Memory: 347M/1818M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.1:analyze-only (default) on project beam-runners-java-fn-execution: Dependency problems found -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :beam-runners-java-fn-execution
STDERR: ++ dirname /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/generate-sources.sh
+ HERE=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8
+ EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java
+ JAVA8_EXAMPLES_ROOT=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8
+ ARCHETYPE_ROOT=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources
+ mkdir -p /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ mkdir -p /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ rsync -a --exclude cookbook --exclude complete /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java --delete
+ rsync -a --exclude cookbook --exclude complete --exclude '*IT.java' /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java --delete
+ rsync -a /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/main/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java
+ rsync -a /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/../../../../examples/java8/src/test/java/org/apache/beam/examples/ /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java
+ find /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^package org\.apache\.beam\.examples/package ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/main/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src/test/java -name '*.java' -print0
+ xargs -0 sed -i.bak 's/^import org\.apache\.beam\.examples/import ${package}/g'
+ find /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/java/maven-archetypes/examples-java8/src/main/resources/archetype-resources/src -name '*.bak' -delete
log4j:WARN No appenders could be found for logger (freemarker.cache).
log4j:WARN Please initialize the log4j system properly.
Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking.
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/gen_protos.py:50: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/.eggs/grpcio-wheels
WARNING:root:Installing grpcio-tools took 6.05 seconds.
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/target/python/lib/python2.7/site-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
2018-01-13 06:08:29,416 3b7012e3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-13 06:08:29,443 3b7012e3 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-13 06:18:29,466 3b7012e3 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-13 06:18:29,477 3b7012e3 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/3b7012e3/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-13 06:18:29,478 3b7012e3 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-13 06:18:29,480 3b7012e3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-13 06:18:29,482 3b7012e3 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-13 06:18:29,536 3b7012e3 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-13 06:18:29,537 3b7012e3 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/3b7012e3/pkb.log
2018-01-13 06:18:29,537 3b7012e3 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/3b7012e3/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #783
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/783/display/redirect?page=changes>
Changes:
[chamikara] Updates the error message when trying to execute an export job on an
[batbat] Fixed a bug that prevented processing time timers to be reset
[batbat] Added ParDoTest for resetting timers.
[tgroh] Handle a populated Error field in FnApiControlClient
[sidhom] Expose side inputs in CombineTranslation and refactor portable
[sidhom] Use portable constructs in Flink batch translator
[lcwik] [BEAM-2863] Add the ability to length prefix unknown coders using the
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b88d15009e8da25c71a08affcbcf74fec7622c91 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b88d15009e8da25c71a08affcbcf74fec7622c91
Commit message: "Merge pull request #4343: [BEAM-2795] Use portable constructs in Flink batch translator"
> git rev-list a5b577930488a6b4b98695d5a4d579ebf023fa03 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6667882912959727711.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6896236520613249839.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4109562814915540895.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4654349215226416719.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins252305640437607359.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
Traceback (most recent call last):
File "PerfKitBenchmarker/pkb.py", line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 869, in Main
SetUpPKB()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 740, in SetUpPKB
vm_util.GenTempDir()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 121, in GenTempDir
temp_dir.CreateTemporaryDirectories()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py",> line 62, in CreateTemporaryDirectories
os.makedirs(path)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/05cc4074'
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user sidhom@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user batbat@batbat-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net
Build failed in Jenkins: beam_PerformanceTests_Python #782
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/782/display/redirect>
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision a5b577930488a6b4b98695d5a4d579ebf023fa03 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a5b577930488a6b4b98695d5a4d579ebf023fa03
Commit message: "Merge pull request #4339 from iemejia/BEAM-3405-maxnumrecords"
> git rev-list a5b577930488a6b4b98695d5a4d579ebf023fa03 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2236605332702765696.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6557817010788116363.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7046439575449041119.sh
+ pip install --user -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4146469967921556685.sh
+ pip install --user -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: hdfs3<0.4.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.3.0.dev0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0.dev0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: requests<3.0dev,>=2.13.0 in /usr/local/lib/python2.7/dist-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0.dev0)
Installing collected packages: apache-beam
Found existing installation: apache-beam 2.3.0.dev0
Uninstalling apache-beam-2.3.0.dev0:
Successfully uninstalled apache-beam-2.3.0.dev0
Running setup.py develop for apache-beam
Successfully installed apache-beam
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins99275596170703928.sh
+ python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.pkb_results --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-01-12 18:00:32,583 9b3ae969 MainThread INFO Verbose logging to: /tmp/perfkitbenchmarker/runs/9b3ae969/pkb.log
2018-01-12 18:00:32,583 9b3ae969 MainThread INFO PerfKitBenchmarker version: v1.12.0-259-g5a6cff9
2018-01-12 18:00:32,584 9b3ae969 MainThread INFO Flag values:
--beam_sdk=python
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-01-12 18:00:33,135 9b3ae969 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-01-12 18:00:33,135 9b3ae969 MainThread INFO Initializing the edw service decoder
2018-01-12 18:00:33,260 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-01-12 18:00:33,263 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-01-12 18:00:33,264 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git
2018-01-12 18:00:39,046 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install -DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-01-12 18:09:16,243 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-01-12 18:09:16,312 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Running: python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9b3ae969/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz
2018-01-12 18:19:16,318 9b3ae969 Thread-4 ERROR IssueCommand timed out after 600 seconds. Killing command "python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9b3ae969/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz".
2018-01-12 18:19:16,326 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Ran python setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=/tmp/perfkitbenchmarker/runs/9b3ae969/beam/sdks/python/target/apache-beam-2.3.0.dev0.tar.gz. Got return code (-9).
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355: UserWarning: Normalizing '2.3.0.dev' to '2.3.0.dev0'
normalized_version,
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9b3ae969/beam/sdks/python/apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
/tmp/perfkitbenchmarker/runs/9b3ae969/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... /tmp/perfkitbenchmarker/runs/9b3ae969/beam/sdks/python/apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
2018-01-12 18:19:16,327 9b3ae969 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 601, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 504, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-01-12 18:19:16,329 9b3ae969 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-01-12 18:19:16,332 9b3ae969 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-01-12 18:19:16,368 9b3ae969 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------
Name UID Status
---------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-01-12 18:19:16,369 9b3ae969 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/9b3ae969/pkb.log
2018-01-12 18:19:16,369 9b3ae969 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/9b3ae969/completion_statuses.json
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user swegner@google.com
Not sending mail to unregistered user arostami@google.com
Not sending mail to unregistered user github@alasdairhodge.co.uk
Not sending mail to unregistered user mariagh@mariagh.svl.corp.google.com
Not sending mail to unregistered user ankurgoenka@gmail.com
Not sending mail to unregistered user ehudm@google.com
Not sending mail to unregistered user kirpichov@google.com
Not sending mail to unregistered user dariusz.aniszewski@polidea.com
Not sending mail to unregistered user jb@nanthrax.net