You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/07/25 15:45:11 UTC
Build failed in Jenkins: beam_PerformanceTests_Python #1253
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1253/display/redirect>
------------------------------------------
GitHub pull request #6056 of commit d951271cb4412facc746bf3aab4b9dcf86259472, no merge conflicts.
Setting status of d951271cb4412facc746bf3aab4b9dcf86259472 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/1253/ and message: 'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/*
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/6056/*:refs/remotes/origin/pr/6056/*
> git rev-parse refs/remotes/origin/pr/6056/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/6056/merge^{commit} # timeout=10
Checking out Revision f5b091a50c6bfa3e7f10e361eab9d4d107763ddc (refs/remotes/origin/pr/6056/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f f5b091a50c6bfa3e7f10e361eab9d4d107763ddc
Commit message: "Merge d951271cb4412facc746bf3aab4b9dcf86259472 into f2a1fd4307929aa938067058205a5ff79dc0207f"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4613836979666378994.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7257108433714162759.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6164015999892067355.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8665091684199027555.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5604433107216637169.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6086223940262032725.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8326416354300863133.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6909867195755979876.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1403573749653444305.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8809190521784436377.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1458877980335576090.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-25 15:38:58,755 2f8cc01e MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/pkb.log>
2018-07-25 15:38:58,756 2f8cc01e MainThread INFO PerfKitBenchmarker version: v1.12.0-729-ga00bd87
2018-07-25 15:38:58,757 2f8cc01e MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-25 15:38:59,028 2f8cc01e MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-25 15:38:59,215 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-25 15:38:59,223 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-25 15:38:59,224 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-25 15:45:10,639 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-25 15:45:10,735 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 198, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 274, in _BuildPythonCommand
raise RuntimeError('No python binary is found')
RuntimeError: No python binary is found
2018-07-25 15:45:10,736 2f8cc01e MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-25 15:45:10,739 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 804, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 87, in SubmitJob
self.spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 198, in BuildBeamCommand
cmd = _BuildPythonCommand(benchmark_spec, classname, job_arguments)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_benchmark_helper.py",> line 274, in _BuildPythonCommand
raise RuntimeError('No python binary is found')
RuntimeError: No python binary is found
2018-07-25 15:45:10,739 2f8cc01e MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-25 15:45:10,794 2f8cc01e MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-25 15:45:10,795 2f8cc01e MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/pkb.log>
2018-07-25 15:45:10,795 2f8cc01e MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2f8cc01e/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Jenkins build is back to normal : beam_PerformanceTests_Python
#1268
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1268/display/redirect?page=changes>
Build failed in Jenkins: beam_PerformanceTests_Python #1267
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1267/display/redirect>
------------------------------------------
[...truncated 100.18 KB...]
2018-08-04 01:09:27,927 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:09:46,098 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:09:49,974 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6d1b59a3091e4529_00000165027bb68d_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6d1b59a3091e4529_00000165027bb68d_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6d1b59a3091e4529_00000165027bb68d_1 ... (0s) Current status: DONE
2018-08-04 01:09:49,975 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:10:12,981 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:10:16,990 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4840b2454a8e3494_00000165027c1fc7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4840b2454a8e3494_00000165027c1fc7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4840b2454a8e3494_00000165027c1fc7_1 ... (0s) Current status: DONE
2018-08-04 01:10:16,991 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:10:44,182 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:10:48,193 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r178283251e23bd57_00000165027c99aa_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r178283251e23bd57_00000165027c99aa_1 ... (0s) Current status: RUNNING Waiting on bqjob_r178283251e23bd57_00000165027c99aa_1 ... (0s) Current status: DONE
2018-08-04 01:10:48,194 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:09,452 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:11:13,329 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4b3530b4d72d9f35_00000165027cfc3b_1 ... (0s) Current status: DONE
2018-08-04 01:11:13,330 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:32,307 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:11:36,404 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2dab7c4da2143862_00000165027d55b3_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r2dab7c4da2143862_00000165027d55b3_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2dab7c4da2143862_00000165027d55b3_1 ... (0s) Current status: DONE
2018-08-04 01:11:36,405 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:11:56,492 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:12:00,468 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6afc3f5f01d3da95_00000165027db417_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6afc3f5f01d3da95_00000165027db417_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6afc3f5f01d3da95_00000165027db417_1 ... (0s) Current status: DONE
2018-08-04 01:12:00,469 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:12:28,107 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:12:32,126 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4c35d1eaaa7322ec_00000165027e2fa0_1 ... (0s) Current status: DONE
2018-08-04 01:12:32,127 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:12:59,230 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:13:03,189 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r56a8555cee86c30e_00000165027ea94b_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r56a8555cee86c30e_00000165027ea94b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r56a8555cee86c30e_00000165027ea94b_1 ... (0s) Current status: DONE
2018-08-04 01:13:03,189 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:13:31,562 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:13:35,580 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4a57b6221bd70572_00000165027f2756_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4a57b6221bd70572_00000165027f2756_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4a57b6221bd70572_00000165027f2756_1 ... (0s) Current status: DONE
2018-08-04 01:13:35,580 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:00,552 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:04,695 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r488e26fae3081563_00000165027f98c2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r488e26fae3081563_00000165027f98c2_1 ... (0s) Current status: RUNNING Waiting on bqjob_r488e26fae3081563_00000165027f98c2_1 ... (0s) Current status: DONE
2018-08-04 01:14:04,695 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:21,638 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:25,674 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r4445d9fb83085a53_00000165027feb2c_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r4445d9fb83085a53_00000165027feb2c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4445d9fb83085a53_00000165027feb2c_1 ... (0s) Current status: DONE
2018-08-04 01:14:25,674 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:14:43,773 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:14:47,831 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r2fc5cc3b0eccf8_0000016502804192_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r2fc5cc3b0eccf8_0000016502804192_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2fc5cc3b0eccf8_0000016502804192_1 ... (0s) Current status: DONE
2018-08-04 01:14:47,832 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:15:09,405 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:15:13,393 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r1da0a31736f5c6b5_000001650280a5c2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r1da0a31736f5c6b5_000001650280a5c2_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1da0a31736f5c6b5_000001650280a5c2_1 ... (0s) Current status: DONE
2018-08-04 01:15:13,394 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:15:42,871 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:15:46,803 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r296782e907644c9c_000001650281286f_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r296782e907644c9c_000001650281286f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r296782e907644c9c_000001650281286f_1 ... (0s) Current status: DONE
2018-08-04 01:15:46,803 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:04,053 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:08,105 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r168cf30e70c68a2_0000016502817b34_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r168cf30e70c68a2_0000016502817b34_1 ... (0s) Current status: RUNNING Waiting on bqjob_r168cf30e70c68a2_0000016502817b34_1 ... (0s) Current status: DONE
2018-08-04 01:16:08,106 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:31,439 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:35,400 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r5df6a78dbcd141cd_000001650281e615_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r5df6a78dbcd141cd_000001650281e615_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5df6a78dbcd141cd_000001650281e615_1 ... (0s) Current status: DONE
2018-08-04 01:16:35,401 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:16:55,147 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:16:59,535 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6a5b38b8cff7b7cb_00000165028242d8_1 ... (0s) Current status: DONE
2018-08-04 01:16:59,536 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:17:18,503 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:17:22,519 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r540d80a10f444f7e_0000016502829e11_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r540d80a10f444f7e_0000016502829e11_1 ... (0s) Current status: RUNNING Waiting on bqjob_r540d80a10f444f7e_0000016502829e11_1 ... (0s) Current status: DONE
2018-08-04 01:17:22,519 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:17:46,148 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:17:50,073 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r41abbeef2f17231a_00000165028309f8_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r41abbeef2f17231a_00000165028309f8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r41abbeef2f17231a_00000165028309f8_1 ... (0s) Current status: DONE
2018-08-04 01:17:50,073 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:18:14,823 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:18:18,648 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r1089e7f2c23201e8_00000165028379fc_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r1089e7f2c23201e8_00000165028379fc_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1089e7f2c23201e8_00000165028379fc_1 ... (0s) Current status: DONE
2018-08-04 01:18:18,649 0baf5535 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code.
2018-08-04 01:18:47,328 0baf5535 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json>
2018-08-04 01:18:51,411 0baf5535 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0baf5535/perfkit-bq-pubhz2Lwj.json}> ReturnCode:1
STDOUT:
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r45ba4d1ae2d72753_000001650283f8e7_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT
STDERR: Upload complete.Waiting on bqjob_r45ba4d1ae2d72753_000001650283f8e7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r45ba4d1ae2d72753_000001650283f8e7_1 ... (0s) Current status: DONE
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/pkb.py",> line 21, in <module>
sys.exit(Main())
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 1012, in Main
return RunBenchmarks()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 939, in RunBenchmarks
collector.PublishSamples()
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 928, in PublishSamples
publisher.PublishSamples(self.samples)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 538, in PublishSamples
vm_util.IssueRetryableCommand(load_cmd)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 249, in WrappedFunction
return f(*args, **kwargs)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 387, in IssueRetryableCommand
'Command returned a non-zero exit code.\n')
perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code.
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1266
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1266/display/redirect>
------------------------------------------
GitHub pull request #6142 of commit c48115558d7af37b518f69463dba5bd7612054a5, no merge conflicts.
Setting status of c48115558d7af37b518f69463dba5bd7612054a5 to PENDING with url https://builds.apache.org/job/beam_PerformanceTests_Python/1266/ and message: 'Build started for merge commit.'
Using context: Python SDK Performance Test
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/6142/*:refs/remotes/origin/pr/6142/*
> git rev-parse refs/remotes/origin/pr/6142/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/6142/merge^{commit} # timeout=10
Checking out Revision abc1bce29360cd14fc5fb6f62c1c52193d112fa4 (refs/remotes/origin/pr/6142/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f abc1bce29360cd14fc5fb6f62c1c52193d112fa4
Commit message: "Merge c48115558d7af37b518f69463dba5bd7612054a5 into 2d3ce5181ad144de9c5cc6f945a6d40f9c0ae795"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8333306446748541054.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker>
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3957460423414083293.sh
+ rm -rf <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env>
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2172882893396186781.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env> --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5967812430427483923.sh
+ virtualenv <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env> --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5622511714784926622.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/.beam_env/bin/pip> install --upgrade setuptools pip grpcio-tools==1.3.5
Requirement already up-to-date: setuptools in ./src/.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./src/.beam_env/lib/python2.7/site-packages (18.0)
Collecting grpcio-tools==1.3.5
Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: grpcio>=1.3.5 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (1.13.0)
Requirement already satisfied, skipping upgrade: protobuf>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (3.6.0)
Requirement already satisfied, skipping upgrade: six>=1.5.2 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.11.0)
Requirement already satisfied, skipping upgrade: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
Requirement already satisfied, skipping upgrade: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (3.2.0)
Installing collected packages: grpcio-tools
Successfully installed grpcio-tools-1.3.5
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins514423414470920267.sh
+ <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/.perfkit_env/bin/pip> install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./PerfKitBenchmarker/.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./PerfKitBenchmarker/.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3743757725069306645.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker>
fatal: destination path '<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker'> already exists and is not an empty directory.
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1265
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1265/display/redirect?page=changes>
Changes:
[yifanzou] [BEAM-5018] Upgrade org.tukaani:xz to 1.8
[ryan.blake.williams] Revert "Install fastavro only in linux"
[ankurgoenka] Tests for running Python on Flink.
[ankurgoenka] Fixing lint issues
[gleb] [BEAM-5050] [SQL] Fix aggregation of nulls
[yifanzou] [BEAM-3906] Automate Validation Aganist Python Wheel
[chamikara] Fixes a Javadoc error
[ankurgoenka] Adding gradle task for python streaming validates flink runner
[aaltay] [BEAM-4391] Example of distributed optimization (#5736)
[tweise] [BEAM-4778] add option to flink job server to clean staged artifacts
[pablo] Fixed wrong isRelease() call in publish
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f96b25a947a58542189e62fe46236b63ea99ca6a (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f96b25a947a58542189e62fe46236b63ea99ca6a
Commit message: "Fixed wrong isRelease() call in publish"
> git rev-list --no-walk f2ae73be3d3d1cb7c3d111505f832aed44bc92eb # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4635927866774441150.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6534342280852594361.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4523054878649429971.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2044191547514904883.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins869372471515896118.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3804156420611226352.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5269395541421208009.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1926765491110891651.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3145344628393015723.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: setuptools>=18.0 in ./.beam_env/lib/python2.7/site-packages (from fastavro==0.19.7->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3715548868458883605.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1264
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1264/display/redirect>
------------------------------------------
[...truncated 1.73 KB...]
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3345867546547775270.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1399758868827400867.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6273576107848702835.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7994866195948922848.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins89803871673548492.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4391940347346608197.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins401290153052960708.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7704870098660130538.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins667339526735438882.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: setuptools>=18.0 in ./.beam_env/lib/python2.7/site-packages (from fastavro==0.19.7->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2076722097612132100.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6171094355052098590.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2526341026312251996.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
2018-08-02 16:57:46,438 83b8abb4 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/pkb.log>
2018-08-02 16:57:46,438 83b8abb4 MainThread INFO PerfKitBenchmarker version: v1.12.0-747-g7e4ea3f
2018-08-02 16:57:46,439 83b8abb4 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-08-02 16:57:46,693 83b8abb4 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-08-02 16:57:46,900 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-08-02 16:57:46,905 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-08-02 16:57:46,905 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-08-02 17:03:48,932 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-08-02 17:03:49,025 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-08-02 18:03:49,033 83b8abb4 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-08-02 18:03:49,043 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-08-02 18:03:49,044 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-02 18:03:49,046 83b8abb4 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-08-02 18:03:49,049 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 808, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-02 18:03:49,049 83b8abb4 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-08-02 18:03:49,079 83b8abb4 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-08-02 18:03:49,079 83b8abb4 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/pkb.log>
2018-08-02 18:03:49,080 83b8abb4 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/83b8abb4/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1263
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1263/display/redirect?page=changes>
Changes:
[rober] Avoid redundant computation when invoking ProcessElements in ParDo
[github] Update fn_test.go
[rober] De-dup pre & post invoke code in pardo.go
[timrobertson100] [BEAM-2661] Adds KuduIO
[github] Properly revert log line changes in fn_test.go
[rober] Add comparitive benchmark between cached and single use invoker.
[github] [BEAM-4852] Only read symbol table when required.
[github] Update symbols.go
[timrobertson100] [BEAM-2661] KuduIO: Add missing licenses
[ehudm] Make with_attributes kwarg optional.
[ehudm] Add decode and encode steps to streaming_wordcount
[qinyeli] Interactive Beam -- display update
[ehudm] Fix PubSubMessageMatcher not acking messages.
[ebeach] [BEAM-4862] Fixes bug in Spanner's MutationGroupEncoder by converting
[herohde] Add utilities for manipulating proto pipelines
[herohde] [BEAM-4636] Refactor Go dataflow runner for more flexible uses
[herohde] [BEAM-4813] Switch to translation from model protos to Dataflow v1beta3
[herohde] Add more Go integration tests and fix CoGBK bugs
[herohde] [BEAM-4636] Ensure unique filename for compiled worker binaries
[herohde] Handle explicit flatten
[herohde] Fix Dataflow argument bug
[herohde] CR: tweak code and fix comments
[aaltay] [BEAM-4751] fix missing pylint3 check for io subpackage (#5916)
[pablo] Add GitScm poll trigger for post-commit tests.
[relax] Enable schemas for Dataflow runner.
[tweise] [BEAM-5023] Send HarnessId in DataClient (#6066)
[relax] Fix BigQuery job retry mechanism.
[relax] Add exponential backoff on retries.
[relax] Apply code-review comment.
[tweise] [BEAM-2930] Side inputs are not yet supported in streaming mode.
[github] Don't rely on order of elements in a PCollection after GBK in
[relax] Properly handle running jobs.
[devinduan] Spelling mistakes
[relax] Convert BeamSQL to use Schemas.
[relax] Deprecate getRowCoder.
[relax] Add setSchema to remaining Table objects.
[relax] Delete a bunch of code that is no longer used.
[relax] Move utilities into schemas.utils package.
[relax] Convert Nexmark to use schemas.
[relax] remove deprecated Schema.getRowCoder
[relax] Apply spotless.
[relax] Plumb schema through DoFn.
[relax] Add helper setRowSchema to eliminatre boilerplate.
[rober] Avoid overwritting user changes to Resolver
[rober] Clean up deferedResolver
[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2
[migryz] Add retries for gpg key query logic.
[pablo] Fix scheduling for jobs
[relax] Add convenience methods for pojo and javabean schema registration.
[relax] Address code-review comments.
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to Flink 1.5.2"
> git rev-list --no-walk ff95a82e461bd8319d9733be60e75992ba90cd7c # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8260919133135875607.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3883146302874079411.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8573027766605549178.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7954528295858307459.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins117727936914663087.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4530960035642880902.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5189163966448436943.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6867585134301396263.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1463111188888185076.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6724119042722330580.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1262
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1262/display/redirect>
------------------------------------------
[...truncated 2.19 KB...]
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins134203476966691273.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266418180377408786.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2763990076871907715.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3751194031371478445.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3369103436413740422.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2139078975145550914.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8720383783791910367.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4964247970169696296.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5972620634519769960.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_HadoopInputFormat/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins168109957690731329.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
creating apache_beam.egg-info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9219101386029762686.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1829006631628808140.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
2018-08-01 11:22:25,420 87a2f212 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/pkb.log>
2018-08-01 11:22:25,421 87a2f212 MainThread INFO PerfKitBenchmarker version: v1.12.0-747-g7e4ea3f
2018-08-01 11:22:25,422 87a2f212 MainThread INFO Flag values:
--beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-08-01 11:22:25,728 87a2f212 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-08-01 11:22:25,960 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-08-01 11:22:25,964 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-08-01 11:22:25,964 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-08-01 11:28:25,618 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-08-01 11:28:25,719 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-08-01 12:28:25,727 87a2f212 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-08-01 12:28:25,737 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-08-01 12:28:25,738 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-01 12:28:25,739 87a2f212 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-08-01 12:28:25,741 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 808, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 672, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 552, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-08-01 12:28:25,742 87a2f212 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-08-01 12:28:25,796 87a2f212 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-08-01 12:28:25,797 87a2f212 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/pkb.log>
2018-08-01 12:28:25,797 87a2f212 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/87a2f212/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1261
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1261/display/redirect?page=changes>
Changes:
[Jozef.Vilcek] Finalize writes before returning filenames via WriteFileResults
[yifanzou] [BEAM-5012] upgrade spring-expression
[aaltay] [BEAM-4747] mkdirs if they don't exist in localfilesystem (#5903)
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ff95a82e461bd8319d9733be60e75992ba90cd7c (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f ff95a82e461bd8319d9733be60e75992ba90cd7c
Commit message: "Merge pull request #6091: [BEAM-5012] upgrade spring-expression"
> git rev-list --no-walk 7ea9c82457c96835215bf799836738e256bac1da # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8865120929856213274.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1473121459185172423.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3734836492793927241.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8679179533584043181.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2159511384415016943.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8080162836156584622.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1309902400704482311.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4478482058959712825.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1441016414319258742.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3075750881756728596.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1260
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1260/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2607446597210156195.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2443095237927008836.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1189694480124475100.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2975676980433889651.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6197281220405623172.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3153161168204468762.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3320282445925404651.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9169151424390991382.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins765699272845885303.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4561233640386505352.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4538288153756298284.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2064771233173343101.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=3600 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-30 10:18:46,224 4159b0b0 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/pkb.log>
2018-07-30 10:18:46,225 4159b0b0 MainThread INFO PerfKitBenchmarker version: v1.12.0-742-g2a71a6d
2018-07-30 10:18:46,226 4159b0b0 MainThread INFO Flag values:
--beam_it_timeout=3600
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-30 10:18:46,492 4159b0b0 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-30 10:18:46,668 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-30 10:18:46,672 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-30 10:18:46,672 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-30 10:24:25,112 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-30 10:24:25,205 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-30 11:24:25,213 4159b0b0 Thread-3 ERROR IssueCommand timed out after 3600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-30 11:24:25,222 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-30 11:24:25,223 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 11:24:25,224 4159b0b0 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-30 11:24:25,226 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 11:24:25,227 4159b0b0 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-30 11:24:25,243 4159b0b0 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-30 11:24:25,244 4159b0b0 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/pkb.log>
2018-07-30 11:24:25,244 4159b0b0 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/4159b0b0/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1259
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1259/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1960843742017956862.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5401992170880531760.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3905576078412088570.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2493266811912257130.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6936278536347873486.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins458815562325241686.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2381364775078793896.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4726826176375157776.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8334416447047675338.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4358843471068673199.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2274858807667533021.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7191424344200540075.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=2400 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-30 08:41:41,842 2c549279 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/pkb.log>
2018-07-30 08:41:41,844 2c549279 MainThread INFO PerfKitBenchmarker version: v1.12.0-742-g2a71a6d
2018-07-30 08:41:41,846 2c549279 MainThread INFO Flag values:
--beam_it_timeout=2400
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-30 08:41:42,216 2c549279 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-30 08:41:42,393 2c549279 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-30 08:41:42,396 2c549279 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-30 08:41:42,397 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-30 08:47:24,989 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-30 08:47:25,083 2c549279 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-30 09:27:25,091 2c549279 Thread-3 ERROR IssueCommand timed out after 2400 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-30 09:27:25,099 2c549279 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-30 09:27:25,100 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 09:27:25,101 2c549279 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-30 09:27:25,103 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-30 09:27:25,104 2c549279 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-30 09:27:25,140 2c549279 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-30 09:27:25,141 2c549279 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/pkb.log>
2018-07-30 09:27:25,141 2c549279 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/2c549279/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1258
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1258/display/redirect?page=changes>
Changes:
[ehudm] Python PubSub updates:
[ehudm] Fix lint errors and clarify PubsubMessage docs.
[ehudm] Skip unit test if pubsub is not installed.
[ehudm] Fix docstring indentation.
[ehudm] Minor fixes.
[lcwik] [BEAM-4860] Ensure that shaded artifacts that are produced don't leak
[pablo] [BEAM-4818] Move the visteg plugin to the root project.
[lcwik] [BEAM-4262] Bumping bigtable versions (#6078)
[github] [BEAM-4658] Update pipeline representation in runner support libraries
[pablo] Changed hardcoded version number into env variable. Added more comments.
[aaltay] [BEAM-4076] Fix schemas on Dataflow and FnApi (#6072)
[qinyeli] Introducing version into CacheManager
[lukasz.gajowy] [BEAM-5037] Fix deserialization problem due to transient field
[lukasz.gajowy] Remove redundant static field
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 7ea9c82457c96835215bf799836738e256bac1da (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 7ea9c82457c96835215bf799836738e256bac1da
Commit message: "Merge pull request #6083: [BEAM-5037] fix synthetic options"
> git rev-list --no-walk aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8055711672218079793.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3631238739723495981.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1463782467337498506.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7762123783080025661.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4373957814379593676.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2690147038434867229.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7250500320601567036.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7762139428107416882.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5165480218991547042.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2566799033419822571.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1257
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1257/display/redirect>
------------------------------------------
[...truncated 1.68 KB...]
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3313402882749527139.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4447330106744310889.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3105004998856343976.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1869377136885516741.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins416301535208885651.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2952868600336155078.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6523644204269782559.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins300810585250411513.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4829542257814454879.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins824765903847337789.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6662012102409384417.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6039706310566305906.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_timeout=1500 --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-27 16:27:40,882 0be68e21 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/pkb.log>
2018-07-27 16:27:40,882 0be68e21 MainThread INFO PerfKitBenchmarker version: v1.12.0-735-ga3f334a
2018-07-27 16:27:40,883 0be68e21 MainThread INFO Flag values:
--beam_it_timeout=1500
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-27 16:27:41,228 0be68e21 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-27 16:27:41,417 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-27 16:27:41,421 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-27 16:27:41,422 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-27 16:33:23,819 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-27 16:33:23,915 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-27 16:58:23,923 0be68e21 Thread-3 ERROR IssueCommand timed out after 1500 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-27 16:58:23,932 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-27 16:58:23,933 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:58:23,934 0be68e21 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-27 16:58:23,937 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:58:23,937 0be68e21 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-27 16:58:23,992 0be68e21 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-27 16:58:23,993 0be68e21 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/pkb.log>
2018-07-27 16:58:23,994 0be68e21 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/0be68e21/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1256
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1256/display/redirect>
------------------------------------------
[...truncated 1.66 KB...]
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3539873326631408448.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8647486184674439503.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1424138257925272438.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5754435381585259556.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2154991725107398410.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2814333558721416384.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3696887024533641845.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4378314190794416191.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8949469932260524169.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8410315169384367550.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8857752751154454891.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1099949695612912981.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-27 16:04:45,694 d1536932 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/pkb.log>
2018-07-27 16:04:45,695 d1536932 MainThread INFO PerfKitBenchmarker version: v1.12.0-735-ga3f334a
2018-07-27 16:04:45,696 d1536932 MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-27 16:04:46,023 d1536932 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-27 16:04:46,220 d1536932 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-27 16:04:46,224 d1536932 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-27 16:04:46,225 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-27 16:10:33,485 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-27 16:10:33,584 d1536932 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-27 16:20:33,592 d1536932 Thread-3 ERROR IssueCommand timed out after 600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-27 16:20:33,601 d1536932 MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-27 16:20:33,602 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:20:33,603 d1536932 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-27 16:20:33,605 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 806, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 670, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 550, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-27 16:20:33,606 d1536932 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-27 16:20:33,620 d1536932 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-27 16:20:33,621 d1536932 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/pkb.log>
2018-07-27 16:20:33,621 d1536932 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/d1536932/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1255
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1255/display/redirect?page=changes>
Changes:
[kedin] [SQL] Remove explicit timestamp parsing from PubsubIOJsonTable
[kedin] [SQL] Wrap Pubsub payload into a message schema
[pablo] Change reading sideinput experimental flag into new pattern.
[mariagh] Call assert_that per window via reify_windows flag
[github] [BEAM-4268] Improving the separation between Metrics API and Execution
[aromanenko.dev] [BEAM-4048] Refactor COMBINE mode for reading/writing from/to Pub/Sub
[matthias] Futurize metrics subpackage
[pablo] Adding context switch to operations missing it.
[sidhom] [BEAM-2597] Flink batch ExecutableStage operator
[thw] Fix compile error in RemoteExecutionTest.
[klk] Move Release Guide to committers section
[klk] Move committer section to the bottom (we can find it :-)
[tgroh] Suppress more flaky tests.
[klk] Streamline the contribution section
[tgroh] Add a FlattenRunnerTest for a multi-consumed input
[tgroh] Use ListMultimap when creating Transform Runners
[tgroh] Migrate ImpulseEvaluatorFactory
[tgroh] Migrate FlattenEvaluatorFactoryTest
[tgroh] Migrate GroupByKeyOnly to the RR
[tgroh] Use ModelCoders in LengthPrefixUnknownCoders
[ekirpichov] [BEAM-3515] Portable translation of SplittableProcessKeyed
[ekirpichov] Fixes compile error in FlinkExecutableStageFunctionTest
[ekirpichov] Sickbays flaky test
[robbe.sneyders] Add py27-lint3 test to gradle.build
[klk] Google has scrapped research, now only AI
[iemejia] Fix error-prone warnings for HCatalogIO
[iemejia] Fix error-prone warnings and multiple cleanups based on static analysis
[lcwik] [BEAM-4280] Prevent DirectStreamObserver from blocking indefinitely if
[tgroh] Enable MapFnRunner to provide full WindowedValues
[tgroh] Implement AssignWindows in the Java SDK Harness
[tgroh] Add a static constructor to GrpcStateService
[tgroh] Log when Handler Delegation Throws
[kirpichov] fromQuery().withTemplateCompatibility() should do the same thing as
[klk] Temporarily suppress validation of ai.google links
[iemejia] Fix error-prone warnings for HBaseIO
[timrobertson100] Fix error-prone warnings for SolrIO
[timrobertson100] Fix error-prone warnings for ElasticsearchIO
[timrobertson100] Fix error-prone and some static analysis warnings in
[mergebot] Add link to portability support sheet
[timrobertson100] Fix error-prone and some static analysis warnings in XmlIO
[pablo] Fix error-prone warnings for file-based-io-tests
[kedin] [SQL] Add integration test for PubsubIO JSON -> SQL
[kedin] [SQL] Fix typos, add rewindowing and retriggering to TestPubsubSignal
[apilloud] [BEAM-3983] Add utils for converting to BigQuery types
[apilloud] [BEAM-3983][SQL] Add BigQuery table provider
[apilloud] [BEAM-4248] Copy enums from com.google.cloud
[kedin] Add SQL postCommit to main javaPostCommit
[wcn] Fix log statement to mark sink as generator of info.
[timrobertson100] Fix error-prone warnings for io/hadoop-common
[mairbek] Make Cloud Spanner transaction id public. Allows to build non-library
[daniel.o.programmer] [BEAM-4277] Adding unshaded jars used by Dataflow to poms.
[tgroh] Revert "Log when Handler Delegation Throws"
[tgroh] Log when Handler Delegation Throws
[tgroh] squash! Revert "Log when Handler Delegation Throws"
[pablo] [BEAM-4249] Fix failing validates container test
[valentyn] Add a Python 3 entry to the list of ongoing projects.
[lcwik] [BEAM-4127] Flink streaming translation for portable pipeline. (#5358)
[herohde] [BEAM-3826] Add missing package comments
[kedin] [SQL] Add support for DQL to PubsubJsonTable
[mergebot] Add Go SDK quickstart and minimal documentation page
[mergebot] Add Go to the Wordcount example
[mergebot] Fix mailing list links and docker image
[pablo] Add a dependency on future.
[timrobertson100] Fix error-prone and some static analysis warnings in
[lukasz.gajowy] [BEAM-4140] Utilize beam_location perfkit flag
[github] Update index.md
[kpeterson] Adding a ValueProvider interface.
[klk] Reorganize SQL documentation into sections
[swegner] Upgrade Byte Buddy and FindBugs annotations dependencies.
[swegner] Fix ErrorProne warnings
[swegner] Remove isTriggerSpecified from WindowingStategy equality checks.
[kpeterson] Fixing checkstyle error.
[lcwik] Add window mapping transform.
[lcwik] Use the coder from the side input PCollection definition by removing the
[lcwik] Remove garbage collection from PValueCache
[pablo] Fixing incorrect assert in dependency_test.py
[kedin] [SQL] Add integration test for Pubsub Json Table DLQ
[github] [BEAM-4300] Fix ValidatesRunner tests in Python: run with same mechanism
[tgroh] Add an IdGenerator class to java/fn-execution
[ringwalt] Improve the TaggedOutput error when a tuple is passed as a tag.
[lukasz.gajowy] [BEAM-214] Add ParquetIO
[lukasz.gajowy] [BEAM-214] Add gradle build in ParquetIO
[lukasz.gajowy] [BEAM-214] Refactore the reads to leverage FileIO
[lukasz.gajowy] [BEAM-214] Use Parquet 1.10.0 version in ParquetIO.Sink
[lukasz.gajowy] [BEAM-214] Add ParquetIOIT
[lukasz.gajowy] [BEAM-214] Update ReadFn to an non deprecated, newest Parquet API
[lukasz.gajowy] [BEAM-214] Simplify the IO by removing redundant PTransforms
[lukasz.gajowy] [BEAM-214] Implement writing slices of bytes in BeamOutputStream
[apilloud] [BEAM-3733] [SQL] Use saffron.properties for charset
[pablo] [BEAM-4331] Disable integrationTest task caching
[tgroh] Add a new DockerEnvironmentFactory Constructor
[pablo] Fixing Seed Job
[samuelw] [BEAM-3776] Fix issue with merging late windows where a watermark hold
[pablo] Add future dependency to the Dockerfile
[apilloud] [SQL] Add jdbc:beam:
[apilloud] [SQL] Use CalciteConnection as context
[axelmagn] Create jobsubmission package for job management.
[axelmagn] Add Job Invocation Interfaces
[axelmagn] Add in-memory implementation of JobService
[pablo] Remove stale TODO comments for adding archetype to java precommit
[pablo] Upgrade Gradle shadowJar plugin to latest.
[github] Mention adding a reviewer in PR template. (#5397)
[kedin] Fix Maven build
[rober] Update generated protos in Go SDK
[herohde] Invoke Go Dataflow integration tests from post-commit
[herohde] Add the Go SDK to the README
[timrobertson100] [BEAM-4342] Enforce ErrorProne analysis in hadoop IO, and static
[timrobertson100] [BEAM-4340] Enforce ErrorProne analysis in file-based-io-tests
[timrobertson100] [BEAM-4341] Enforce ErrorProne analysis in google-cloud-platform IO
[timrobertson100] [BEAM-4335] Enforce ErrorProne analysis in amazon-web-services IO
[timrobertson100] [BEAM-4339] Enforce ErrorProne analysis in elasticsearch IO
[timrobertson100] [BEAM-4355] Enforce ErrorProne analysis in XML IO
[timrobertson100] [BEAM-4338] Enforce ErrorProne analysis in common IO
[timrobertson100] [BEAM-4337] Enforce ErrorProne analysis in cassandra IO
[timrobertson100] [BEAM-4355] Reduces scope of findbugs annotations to build time only
[timrobertson100] [BEAM-4353] Enforce ErrorProne analysis in solr IO
[timrobertson100] [BEAM-4345] Enforce ErrorProne analysis in JDBC IO
[timrobertson100] [BEAM-4336] Enforce ErrorProne analysis in AMQP IO
[timrobertson100] [BEAM-4347] Enforce ErrorProne analysis in kafka IO
[dariusz.aniszewski] [BEAM-4041] Increase timeout for getting K8s LoadBalancer external IP
[timrobertson100] [BEAM-4346] Enforce ErrorProne analysis in JMS IO
[timrobertson100] [BEAM-4352] Enforce ErrorProne analysis in Redis IO
[lcwik] [BEAM-4271] Support side inputs for ExecutableStage and provide runner
[daniel.o.programmer] [BEAM-2937] Add new Combine URNs.
[tgroh] Update worker_id Documentation
[robbe.sneyders] Futurize internal subpackage
[dariusz.aniszewski] fixed typo
[mariagh] Add validation for streaming wordcount (streaming_wordcount_debugging)
[pablo] Increasing the concurrent test execution count (#5408)
[tgroh] Add a Default RemoteEnvironment Implementation
[sidhom] Require explicit IdGenerator for all DockerEnvironmentFactory
[sidhom] [BEAM-4148] Add pluggable UrlFactory to ServerFactory
[sidhom] [BEAM-4267] Docker-based JobBundleFactory
[tgroh] Add a Window-only State Context
[lcwik] Address Ben's comments.
[github] Make hash function in Coder base class more conservative.
[github] updated right documentation for HCatalogIO.write
[sidhom] Fix DataStreamsTest checkstyle error
[ankurgoenka] Updating python container version to beam-master-20180518
[wcn] Fix the test to properly run.
[ankurgoenka] Reverting fnapi container version
[ankurgoenka] Copyting dependency.py and dependency_test.py
[ankurgoenka] reformatting code
[ankurgoenka] Introducing classes
[ankurgoenka] gruping public and non public methods
[ankurgoenka] Renaming method and updating reference for file_copy and file_download
[ankurgoenka] Removing Google specific code and unused code in stager.
[ankurgoenka] Removing reference to Google/GCS/Dataflow and restructuring a bit of
[ankurgoenka] Fixing Stager Tests
[ankurgoenka] Making dependency.py use stager.py and fixing dependency_test.py test
[ankurgoenka] Moving stager to portability
[ankurgoenka] Removing stage_job_resources from dependencies.py and other minor
[ankurgoenka] Merging filehandler in stager
[ankurgoenka] renaming GCSStager to _ParameterizedStager
[ankurgoenka] Making a few methods static and moving code around
[ankurgoenka] Applying changes from rebase
[szewinho] HDFS large cluster configuration. Jenkins job updated to use large
[ankurgoenka] Fixing lint
[iemejia] [BEAM-4343] Enforce ErrorProne analysis in HBaseIO
[iemejia] [BEAM-4344] Enforce ErrorProne analysis in the HCatalogIO
[tgroh] Add a BundleFactoryOutputReceiverFactory
[tgroh] Add a DirectJobBundleFactory
[tgroh] Fix Copyright Headers in sdks/java/fn-execution
[yifanzou] rollback BEAM-3256 add archetype testing to gradle build PreCommits
[ankurgoenka] Adding ArtifactStagingFileHandler to push files to staging server
[ankurgoenka] Adding filesystem based artifact service to ULR job service
[ankurgoenka] Adding context manager to ArtifactStagingFileHandler
[ankurgoenka] Converting ArtifactServiceFileHandler to PortableStager based on
[ankurgoenka] Simplifying numeric constants
[ekirpichov] [BEAM-4375] Replaces usage of JUnit runners that do not work well with
[matthias] Futurize options subpackage
[szewinho] [BEAM-4065] Basic performance tests analysis added.
[szewinho] Fixed typo in env name, added unit tests for create report logic, slack
[szewinho] Jenkins job will run tests before executing script. Extended testing of
[timrobertson100] [BEAM-4347] Improve ErrorProne analysis in kafka IO
[timrobertson100] [BEAM-4348] Enforce ErrorProne analysis in kinesis IO
[timrobertson100] [BEAM-4349] Enforce ErrorProne analysis in mongodb IO
[timrobertson100] [BEAM-4351] Enforce ErrorProne analysis in mqtt IO
[lcwik] [BEAM-4060] Add the pom.xml and pom.properties file to the artifact when
[github] [BEAM-1755] Add a directory with build-specific scripts to clear up the
[ankurgoenka] Logging and naming changes
[ankurgoenka] Enhancing test cases
[tgroh] Add an InProcessEnvironmentFactory
[timrobertson100] [BEAM-4348] Removes double-checked locking to aid readability
[ankurgoenka] Updating the dostring
[tgroh] Update InProcessManagedChannelFactory
[tgroh] Add a RemoteStageEvaluatorFactory
[timrobertson100] [BEAM-4354] Enforce ErrorProne analysis in tika IO
[iemejia] Fix build with Maven (Again!)
[lukasz.gajowy] [BEAM-214] Apply reviewer's suggestions
[timrobertson100] [BEAM-4347] Minor changes following PR review
[timrobertson100] [BEAM-4347] Correction of typo following PR review
[tgroh] Use WindowOnlyContext in ReduceFnContextFactory
[tgroh] Update GroupAlsoByWindowEvaluatorFactory
[tgroh] Move InProcessEnvironmentFactory to java-fn-execution
[klk] Autoformat sdks/java/extensions/sql
[klk] Add spotless config to global gradle file
[klk] Activate spotless for SQL module
[boyuanz] Solve BEAM-4393: Missing copyright notices for shaded packages
[iemejia] Remove repeated GCPIO dependency in SQL module (Again!)
[iemejia] Enforce error-prone in ParquetIO, fix missing javadoc and refine deps
[katarzyna.kucharczyk] Replaced created datetime to jenkins build id in kubernetes namespace.
[schroederc] Use ElementEncoders for beam.Create
[boyuanz] Change bulid rules of sdks/java/harness to make harness as a uber-jar
[swegner] Remove DoFnTester from testing guide.
[robertwb] Slowly ramp up wait time in state sampler.
[tgroh] Move the Local Artifact Services to the DirectRunner
[tgroh] Migrate the ReferenceRunnerJobServer to runners-direct
[tgroh] Fix License Headers
[schroederc] Add test case for proto oneof fields in create_test.go
[tgroh] Add a test to show Flatten Execution
[apilloud] [SQL] Fix transitive dependencies
[apilloud] [SQL] Run tests against actual jar
[boyuanz] [BEAM-4393]: Copy root repo's LICENSE & NOTICE into shadowJar
[herohde] Tweak Go integration test driver
[tgroh] Update Docker Command
[iemejia] [BEAM-4306] Enforce ErrorProne analysis in apex runner
[daniel.o.programmer] [BEAM-4406] Updating Java portable environment major version.
[daniel.o.programmer] [BEAM-4406] Updating Python portable environment major version.
[ehudm] Add a custom _url_dirname for local filesystems.
[tgroh] Add a Portable TransformEvaluatorRegistry
[tgroh] Consistent Parameter Order in Evaluator Factories
[tgroh] Make ImmutableExecutableStage Public
[mergebot] Fix typo in becoming-a-committer docs
[iemejia] Fix broken build (again third time this week yay!)
[timrobertson100] [BEAM-4361] Document usage of HBase TableSnapshotInputFormat
[jbonofre] Fix Maven parent version in ParquetIO
[iemejia] [BEAM-4312] Enforce ErrorProne analysis in gearpump runner
[mergebot] Add Apache Kudu IO to the planned IOs table
[mergebot] Add next Java LTS to the WIP section of the contribution guide
[mergebot] Add PubSubIO to Python IO table row
[iemejia] [BEAM-4307] Enforce ErrorProne analysis in
[mergebot] Change ParquetIO status in built-in I/O section
[iemejia] [BEAM-4308] Enforce ErrorProne analysis in runners-core-java
[ammons] Say that merging windows are not supported in Distinct.java.
[lcwik] [BEAM-4411] Update Jackson version to latest to pick up security fixes.
[kenn] Implement UNNEST operator for SQL
[iemejia] [BEAM-4317] Enforce ErrorProne analysis in runners/reference/java
[iemejia] Refine dependencies, make new ones explicit and minor maven plugins
[iemejia] [BEAM-4323] Enforce ErrorProne analysis in sketching extensions
[iemejia] [BEAM-4324] Enforce ErrorProne analysis in sorter extensions
[iemejia] [BEAM-4321] Enforce ErrorProne analysis in join-library extensions
[kenn] Add equals, hashCode, and structuralValue to Nexmark model objects
[thw] [BEAM-4297] Streaming executable stage translation and operator for
[thw] [BEAM-4297] Add serialization test and make output mapping stable.
[lcwik] Add link to design docs and template to the contribution guide.
[timrobertson100] [BEAM-4389] Enable partial updates in ElasticsearchIO
[kenn] Force load of Calcite JDBC driver (avoids internal error)
[kenn] Explicitly load Beam JDBC driver before test
[iemejia] [BEAM-3813] Support (de)serialization of S3 encryption options via JSON
[pablo] [BEAM-4426]: Addressed DataflowDistributionAccumulatorTest failure on
[aaltay] Support installing Beam SDK from a wheel distribution in SDK containers.
[cademarkegard] fix errorprone annotations and enable failOnWarning
[lukasz.gajowy] Add Performance Testing to "Work in Progress" section
[iemejia] Update postgres version to 42.2.2 (Java 8)
[coheigea] Update README to include virtualenv in the build instructions
[iemejia] [BEAM-4356] Enforce ErrorProne analysis in nexmark
[kenn] Finish RowType -> Schema rename
[kenn] Add convenience aliases for nullary type constructors
[kenn] Add MAP field support to Schema.Builder
[kenn] Add static factories for array, row, map types
[kenn] Schema fields are non-null by default
[kenn] TestUtils.RowsBuilder to use FieldType instead of TypeName
[kenn] Clean up CalciteUtils conversions a bit
[pablo] enable failOnWarning for build-tools
[apilloud] Run everything with shadowJar
[apilloud] guava is on kinesis public API
[apilloud] Fix direct runner shadow config
[kenn] Add more field conveniences to Schema.Builder
[kenn] Remove extraneous RowSqlTypes
[pablo] [BEAM-4322] Enforce ErrorProne analysis in protobuf extensions project
[apilloud] Run test on shadowJar on a flag
[kenn] Remove Schema.TypeName.type()
[kenn] Remove unsafe builder methods from FieldType
[apilloud] [SQL] Build mega jar for JDBC
[apilloud] [SQL] JDBC Class Loader IT
[ankurgoenka] proto changes to support artifact_staging_id
[ankurgoenka] Doc changes
[ankurgoenka] Renaming artifact_staging_id to staging_session_token
[ankurgoenka] Fixing python code for staging_session_token proto changes
[marek.simunek] [BEAM-4368] add euphoria java 8 dsl documentation section
[coheigea] Avoid fully qualified class names, when there is an existing (static)
[coheigea] Fixing ErrorProne MissingOverrides warnings
[kedin] [SQL] Fix PubsubsJsonIT
[apilloud] [SQL] Init compiler factory with JDBC class loader
[pablo] clean generated files prior to lint
[ankurgoenka] Fixing java code for staging_session_token proto changes
[ankurgoenka] Go proto update
[amaliujia] Add LIKE operator to Beam SQL.
[kedin] [SQL] Replace planner.compilePipeline() with sqlEnv.parseQuery()
[kedin] [SQL] Add sqlEnv.executeDdl()
[kedin] [SQL] Make planner package-private
[kedin] [SQL] Add factory methods to BeamSqlEnv
[kedin] [SQL] Rename ReadOnlyTableProvider
[coheigea] Fixing some ErrorProne warnings
[github] print() is a function in Python 3
[cademarkegard] [BEAM-4303] Enforce ErrorProne analysis in examples project
[boyuanz] Address BEAM-4328: :beam-sdks-java-io-google-cloud-platform:test failure
[kedin] [SQL] Wrap SQL parsing exceptions in ParseException
[tgroh] Add an abstraction for State and Timers
[tgroh] DirectRunner Cleanups
[tgroh] Link up the Portable DirectRunner
[tgroh] Reuse ID Generators across Environments
[tgroh] Use the Current Users Container as the Environment
[aromanenko.dev] [BEAM-4421] Fix for issue with reading s3 files using ParquetIO
[github] Fix flaky comparison in log_handler_test.py
[tgroh] Use the Current Users Container as the Environment
[tgroh] Enable Docker in the PortableDirectRunner
[tgroh] Move DirectJobBundleFactory to fn-execution
[tgroh] Cleanups to the PortableDirectRunner
[tgroh] Cleanups and Wiring for the ReferenceRunnerJobService
[amyrvold] [BEAM-4253] Update dataflow worker to fix ParDoTest fails in Dataflow
[kedin] [SQL] Add ParseException to sqlEnv.explain()
[kedin] [SQL] Add integration tests for BigQuery writes
[kedin] Add TestBigQuery rule
[tgroh] Use an UnsupportedArtifactRetrievalService
[altay] Move assert_that's new keyword argument to the end
[jhsueh] [BEAM-4320] Enforce ErrorProne analysis in jackson extensions project
[jiangkai] introduce optimized logic plan
[jiangkai] update
[jiangkai] fix unit tests failed
[iemejia] [BEAM-4310] Enforce ErrorProne analysis in
[jiangkai] address comments
[relax] Now that Dataflow is updated, we can finish updating tests and usages to
[szewinho] [BEAM-3214] Add integration test for HBaseIO.
[iemejia] [BEAM-4137] Remove MongoDB specific options from
[ankurgoenka] Code review comments fixes
[ajamato] Add new metrics protos based on s.apache.org/beam-fn-api-metrics
[jiangkai] separate rules into sets
[amaliujia] Address comments: 1. Use SqlFunction.like to implement like operator. 2.
[apilloud] [SQL] Use AutoService for jdbc
[apilloud] [SQL] AutoService for TableProvider
[apilloud] [SQL] Don't cache IT results
[apilloud] [SQL] Test that TextIO and Direct Runner work
[apilloud] [SQL] Add context class loader hack
[rober] [BEAM-4276] Add combiner lifting support to Go SDK
[rober] fixup! Address comments.
[herohde] [BEAM-4474] Ensure unbounded Dataflow jobs are submitted as streaming
[herohde] Fix build break due to artifact staging change
[relax] Add a test with ReferenceRunner
[aromanenko.dev] Added design docs page
[aromanenko.dev] Added first intro sentence.
[aromanenko.dev] Moved one doc to other section and added several new docs
[aromanenko.dev] Added more docs
[github] [BEAM-4480] Fixed deprecated method invoking
[lcwik] [BEAM-4481] Remove duplicate definitions of dependencies.
[herohde] [BEAM-4475] Add container boot code to Go precommit
[herohde] [BEAM-4475] Add Go submodule build to Go precommit
[axelmagn] Fix ProvisionInfo in DockerJobBundleFactory
[github] Improve default value for experiments set in RuntimeValueProvider
[cademarkegard] [BEAM-4326] Enforce ErrorProne analysis in the fn-execution project
[alan] [BEAM-4423] Mark pull requests stale after 60 days; close 7 days after
[jbonofre] Moving to 2.6.0-SNAPSHOT on master branch
[lukasz.gajowy] [BEAM-4415] Add jenkins job for HDFS ParquetIOIT
[lukasz.gajowy] [BEAM-4137] Extract file-related options from IOTestPipelineOptions
[lukasz.gajowy] [BEAM-4137] Extract postgres related options from IOTestPipelineOptions
[timrobertson100] [BEAM-4311] Enforce ErrorProne analysis in Flink runner
[github] [BEAM-4517] Add maven status in README
[jiangkai] add expression boolean casting
[coheigea] Removing extraneous whitespace around the equals operator
[katarzyna.kucharczyk] Changed a namespace to parsed job name with current job number.
[jiangkai] add boolean type for agg function
[jiangkai] support EXISTS operator
[github] Update Environments.java
[github] Remove unneeded collection import.
[github] Combine immutability type fixes. (#3)
[github] fixup!
[lcwik] BEAM-3876 avoid NPE if checkpoint is null in an unbounded source
[lcwik] testing npe fix and exception rethrow in unit tests
[lcwik] Update https://github.com/apache/beam/pull/4894 to correspond with
[github] [BEAM-4276] Add missing scope line
[jiangkai] explain plan unit tests
[lcwik] [BEAM-4517] Add PyPI status badge.
[aljoscha.krettek] [BEAM-3905] Update Flink Runner to Flink 1.5.0
[kedin] [SQL] Manually read rows in TestBigQuery
[robertwb] [BEAM-3981] Cleanup of coders futurization.
[robertwb] Use ByteCountingOutputStream for estimate_size.
[kedin] Add polling assertion support to TestBigQuery
[kedin] [SQL] Add Pubsub to BigQuery E2E integration test
[sidhom] Add reference to ArtifactSourcePool implementation
[sidhom] Clean up FlinkExecutableStageFunction and tests
[sidhom] [BEAM-4523] Implement batch flink executable stage context
[kedin] [SQL] Delete BeamSql
[kedin] [SQL] Rename QueryTransform to SqlTransform
[mergebot] Add SQL area to ongoing projects
[robertwb] Reject unknown arguments in WindowInto.
[mergebot] [BEAM-4488] How to install Python SDK from source [BEAM-4489] Document
[mergebot] More changes
[mergebot] updates after review
[mergebot] Updating release guide to use Gradle commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Addressing comments. Improving commands.
[mergebot] Update release guide for publishing source zip from GitHub
[mergebot] Addressing TODOs
[mergebot] Address comments.
[mergebot] Improving.
[mergebot] Add more updates
[mergebot] More updates
[mergebot] Addressed Alan's comments
[mergebot] Add instructions about how to stage python binaries
[mergebot] copy renamed zi[ file to sdks/python/staging subdir
[mergebot] Update gradle publish with --no-parallel --no-daemon
[kenn] Suppress immutability warning caused by LoadingCache in enum instance
[kenn] Make BeamSqlExpression for operators, use it for string operators
[ekirpichov] Add retrieval_token to GetManifestRequest
[ekirpichov] Refactor code that accesses staging_token
[ekirpichov] Regenerate go protobufs
[kenn] Replace project and filter with calc
[kenn] Disable CalcRemoveRule
[szewinho] Improve consistency of hadoop and hbase scripts, fix shellcheck issues
[iemejia] Fix build (missed auto service for SQL) and model-fn-execution on Flink
[iemejia] [BEAM-4318] Enforce ErrorProne analysis in Spark runner
[wcn] Encode position ids.
[robertwb] Explain why module namespace is modified in Cython codepath.
[apilloud] [SQL] Make getString more generic
[ankurgoenka] Beam File System based Artifact Staging Service
[ankurgoenka] BeamFileSystemArtifactStagingService Tests
[ankurgoenka] Enhancing test case and error handling.
[ankurgoenka] Enhancing testcases and incorporating review comments.
[ankurgoenka] Review comments Fix
[ankurgoenka] Rebasing
[apilloud] [SQL] Support unquoted table types
[herohde] Remove new Go precommit dependencies due to failures
[jiangkai] address conflicts and comments
[jiangkai] remove volcano plan function
[klk] [SQL] Separate assertions for each expression in ExpressionChecker
[klk] Remove StringOperatorsTest in favor of SQL syntax tests
[mergebot] Add Python Kafka to list of in-progress I/O transforms.
[pablo] Upgrade Gradle to version 4.8
[ehudm] Remove with_attributes keyword from ReadFromPubSub.
[xiliu] [BEAM-3079] Update samza-runner with more features and improvements
[xiliu] Add missing apache headers
[swegner] Use new DSL methods for generated POM metadata
[lcwik] [BEAM-4540] Migrate junit/hamcrest to provided scope.
[swegner] [BEAM-925] Fix FindBugs violations in Dataflow Runner and re-enable
[swegner] [BEAM-4313] Fix ErrorProne warnings and enable validation in Dataflow
[ankurgoenka] Followup PR for #5591
[swegner] Add support for pre-commit trigger paths
[swegner] Set path triggers for existing pre-commit test jobs
[rober] Add graph.Fn encoding functions.
[robinyq] Defer calling formatTimestamp() to achieve better performance
[ccy] [BEAM-4549] Use per-pipeline unique ids for side inputs in
[altay] Explain experimental/deprecated from users perspective.
[jiangkai] call SqlFunctions in numeric type conversion
[github] Update Dockerfile for new setup.py changes.
[robbe.sneyders] Futurize utils subpackage
[iemejia] [BEAM-4551] Update spark runner to Spark version 2.3.1
[iemejia] Fix maven build error on sdks/java/io/google-cloud-platform module
[robinyq] Throws exception directly instead of calling checkState()
[github] Remove GPL findbugs dependency (#5609)
[lcwik] Clean-up BeamFnDataBufferingOutboundObserver since Dataflow has migrated
[jiangkai] unit test for BuildExpression literal casting
[aaltay] Futurize portability subpackage (#5385)
[altay] Futurize unpackaged files
[altay] resolved six.string_types equivalency
[altay] Futurize testing subpackage
[altay] Futurize tools subpackage
[altay] Remove old_div
[swegner] Revert "Merge pull request #5611: [BEAM-4445] Filter pre-commit
[robertwb] [BEAM-4546] Implement hot key fanout for combiners.
[robertwb] Use discaring mode for first level of combine.
[echauchot] [BEAM-4283] Export nexmark query execution times to BigQuery
[echauchot] [BEAM-4283] Make BigQueryIO Services classes public and experimental and
[echauchot] [BEAM-4283] Fix PerfsToBigQueryTest and serialization issue in
[echauchot] Add Etienne Chauchot's ongoing work to the contribute page.
[fvoichick] Fixed link to nonexistent IsBounded.apply method
[fvoichick] Fixed documentation of default window size in example code
[fvoichick] Fixed inappropriate Javadoc comments
[aaltay] [BEAM-4333] Add integration tests for python mobile games (#5630)
[jiangkai] ignore explain unit test
[kedin] [SQL] Add INSERT tests to JdbcDriverTest
[kedin] [SQL] Add complex type test to JdbcDriverTest
[kedin] [SQL] Make field names case-sensitive
[robertwb] Suppress warning.
[robertwb] Implement BeamFileSystemArtifactSource
[robertwb] Add job submission capabilities to Flink runner.
[robertwb] Update to new JobInvoker/Invocation interface.
[robertwb] Remove references to Flink distributed file cache.
[robertwb] A bit more documentation.
[robertwb] Address comments on Flink code.
[kirpichov] Removed some dead code, fixed some warnings
[kirpichov] Adds tracing of watermarks in direct runner
[kirpichov] Fixes typo in class name
[kirpichov] Simplifies GBK expansion code a bit
[kirpichov] Fixes bug: Processes values arriving for a GBK key sequentially
[kirpichov] Introduces BundleProgressHandler.
[swegner] Fix ErrorProne violations in direct runner and enable failOnWarnings
[robertwb] [BEAM-4565] Fix hot key fanout in the face of combiner lifting.
[robertwb] Remove trailing whitespace.
[kirpichov] [BEAM-4473,BEAM-4281] Makes ReferenceRunner use a Direct stream observer
[ccy] [BEAM-4549] Revert #5623
[kirpichov] [BEAM-4291] Implements distributed artifact retrieval
[kirpichov] Deletes ArtifactSource
[thw] Fix Flink runner Maven build.
[ccy] [BEAM-4549] Fix side inputs for streaming pipelines on Dataflow
[mergebot] [BEAM-2852] Update Nexmark documentation with Kafka support
[mergebot] [BEAM-4479] Fixed document for Coder
[swegner] Fix ErrorProne violations in fn harness and enable failOnWarnings
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[pablo] Improving state sampler Cython annotations.
[pablo] Adding msecs tracking to side inputs iterator
[pablo] Adding the sampling thread to side input microbench.
[pablo] Adding flag to mask change
[pablo] Fixing all lint raising-format-tuple warnings.
[github] Make build_rules.gradle version of shadow match build.gradle
[swegner] Temporarily exclude website sources from RAT
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[ehudm] Increase Jenkins build retention from 14 to 30 days.
[ankurgoenka] Fixing concurrency bug in BeamFileSystemArtifactServicesTest
[iemejia] Fix Maven build 7
[iemejia] [BEAM-4570] Use pagination in
[thw] Fix ValidatesRunner_Apex #5604
[iemejia] [BEAM-4399] Get Cassandra options out of IOTestPipelineOptions
[iemejia] [BEAM-4399] Refactor CassandraIOIT to be writeThenRead
[iemejia] [BEAM-4399] Update Cassandra driver to version 3.5.0
[kenn] Fix name of heuristic JAVA_CLASS_TO_TYPENAME
[kenn] [SQL] Allow explicit return type in ExpressionChecker
[kenn] DSL-level tests for standard operators
[kenn] Add test that all supported operators are tested
[kenn] [SQL] Make parse failure slightly more actionable
[pablo] Upgrading python dependency on httplib2
[kenn] Fix case sensitivity in JdbcDriverTest
[ehudm] Jupyter notebook for monitoring precommit job times
[lcwik] [BEAM-4576] Add hamcrest dependencies to this scope that was moved to
[mergebot] Add link to @RequiresStableInput design doc
[matthias] Futurize examples subpackage
[apilloud] [SQL] Add parse code for SET
[apilloud] [SQL] Plumb PipelineOptions through BeamRelNode
[apilloud] [SQL] Plumb PipelineOptions through BeamSqlCli
[apilloud] [SQL] Remove unused BeamIOType
[ccy] [BEAM-4582] Fix streaming Create transform on Dataflow
[kirpichov] Propagates artifact retrieval token in Flink runner and to the Java
[ankurgoenka] load beam fileSystems in FnHarness
[axelmagn] Implement new staging token contract in java SDK
[robertwb] Add Impulse primitive to Python SDK
[robertwb] Expand Read to Impulse | ParDo(Split) | ParDo(Read) in Python SDK
[robertwb] Fix default docker image name.
[kirpichov] Add an AddHarnessIdInterceptor
[kirpichov] Remove the runners/reference InProcessManagedChannelFactory
[kirpichov] Allow ManagedChannelFactories to attach Interceptors
[kirpichov] Populate and Use Harness IDs in the Java Harness
[apilloud] [SQL] Set application name to BeamSql
[apilloud] [SQL] Correctly represent pipeline
[apilloud] [SQL] Cleanup table IO functions
[apilloud] [SQL] Simplify overly verbose names
[apilloud] [SQL] Add each Calcite node to Beam graph once
[apilloud] [SQL] Seperate parse from pipeline construction
[swegner] fixup! Fix ErrorProne violations in fn harness and enable failOnWarnings
[kirpichov] Fixes checkstyle
[chamikara] Fixes broken links for downloading source releases.
[lcwik] [BEAM-4592] Make Dataflow understand kind:varint as a well known since
[chamikara] Fixes null pointer exception on empty pubsub message data (#5016)
[kirpichov] A couple more trivial changes extracted from #5662
[lcwik] Update Dataflow container images
[katarzyna.kucharczyk] Created static retry method which accepts retriable method, amount of
[robertwb] Update impulse-based read.
[kenn] Make BeamRelNode take a PCollectionList<Row> always
[rober] Fix build break from PR 5676, which modified method signatures.
[migryz] Implemented working prototype for PostCommit jobs
[migryz] Update post commit jobs to utilize new helper methods
[ankurgoenka] Fix cache references in portable flink runner
[robertwb] [BEAM-4150] Extract IO coders from the RemoteGrpcPort.
[pablo] Adding ignore parameters for gitattributes
[swegner] Remove submodule
[robertwb] Fix and test global combine with fanout.
[migryz] Fix typo in _ghprb suffix
[migryz] Update suffix to be PullRequest
[ccy] Revert #5689 to fix build
[ankurgoenka] Remove unused import
[github] Make suffix shorter
[kirpichov] Make ImmutableExecutableStage constructors public
[kirpichov] [BEAM-4285] Implement Flink batch side input handler
[ankurgoenka] Fixing key lookup in FlinkBatchSideInputHandlerFactory.SideInputHandler
[ekirpichov] Introduces PipelineValidator that checks the well-formedness of a
[cademarkegard] [BEAM-4325] Enforce ErrorProne analysis in the SQL project
[echauchot] [BEAM-4283] Fix naming of the BigQuery fields
[coheigea] Removing some null checks, where we already know that the variable in
[iemejia] [BEAM-3314] Set correctly host and port on RedisIO
[iemejia] [BEAM-3314] Fix error-prone warnings and add extra test for Read
[matthias] Futurize direct runner
[matthias] Futurize dataflow,experimental,job,portability and test subpackages of
[matthias] Futurize complete runners subpackage
[matthias] Cleanup futurize runners
[lcwik] Revert "Revert #5689 to fix build"
[github] Add migration notice to apache/beam website README.me
[robertwb] Always consider Read to be a primitive for dataflow pipelines.
[migryz] Point gearpump to master
[robertwb] [BEAM-4605] Remove subtransform of GBK.
[robbe.sneyders] Futurize typehints subpackage
[kenn] Migrate build_rules.gradle to buildSrc plugin
[amaliujia] support unbounded limit.
[aaltay] Fix backlog bytes reported by CountingSource. (#5719)
[mairbek] Fixed utf8 encoding in Cloud Spanner mutation encoder
[robertwb] Don't require a staged SDK for the Python portable worker.
[pablo] Enabling gradle build to receive extra buildscript repositories
[kenn] Remove build_rules.gradle
[daniel.o.programmer] [BEAM-2937] Comment Combine Grouped Values URN.
[daniel.o.programmer] [BEAM-2973] Moving CombineValues URN to different enum.
[robertwb] Add back old code path for dataflow.
[klk] Fix errors in BeamModulePlugin perf test section
[mergebot] Adds a dependencies guide to Beam Website
[iemejia] Fix maven compile for the 8th time (yay!)
[iemejia] Fix errorprone warnings on extensions/google-cloud-platform-core module
[iemejia] Enable errorprone globally
[klk] Fix Gearpump validatesRunner gradle config
[robertwb] Stage empty manifest if a staging session is given.
[migryz] [BEAM-4559] Add latest post-commit tests status to GH PR
[sidhom] Add registrars for the portable runner
[sidhom] Add endpoint logging for PortableRunner
[github] dataflow_runner.py: use the pbegin's pipeline
[github] materialize_test.go: delint: non-idiomatic block
[klk] Fix docker gated code paths
[altay] Add instructions about how to perform release validations
[ehudm] Parallelize Java precommits
[scott] Spread jenkins load by randomizing daily cron schedule
[amaliujia] logical operator tests
[kirpichov] Adds support for SDF in ULR and the Java SDK.
[klk] Allow specifying delimiter in TextIO
[robbe.sneyders] Address PR comments
[klk] Migrate some date expressions to operators
[klk] Use label "stale" when closing stale PRs
[kenn] Add support for "lines" format text tables
[kenn] Beam SQL shell prompt is now 'BeamSQL>'
[kenn] Override SqlLine.properties to customize welcome and help messages
[pablo] Improving documentation for Metrics results
[xiliu] Finalize Samza Runner with gradle and findbugs issues
[amaliujia] add comparison operator test
[ekirpichov] Address easy review comments
[xiliu] Fix a unit test timeout and minor issues in pom.xml
[kirpichov] Handles memoization of encoded key/window in StateAccessor
[kirpichov] Simplifies API of WireCoders and LengthPrefixUnknownCoders.
[sidhom] [BEAM-4615] Flink job server wrapper and shadow jar
[apilloud] [SQL] Don't wait on Unbounded PCollections
[kirpichov] Address comments
[xiliu] [BEAM-4640] Samza runner postcommit ValidatesRunner job
[aaltay] [BEAM-4333] Add integration tests for python mobile games (#5747)
[jiangkai] [BEAM-4602][BEAM-4598] Date type comparison
[github] Cythonizing a few hot calls (#5733)
[daniel.o.programmer] [BEAM-3708] Adding Combine component implementations to Java SDK
[iemejia] Fix maven build part 9
[xiliu] Remove unused previous names
[daniel.o.programmer] Quick fix, removing commented out code.
[altay] Publish 2.5.0 release
[altay] Change download link for 2.4.0 version to use the archive link
[altay] improvements to fanout
[yifanzou] [BEAM-4302] Add Jenkins job to check Beam dependencies and generate
[ekirpichov] Fixes Flink ValidatesRunner and a compile error
[scott] Split pre-commit tests into separate jobs based on trigger condition.
[scott] Filter pre-commit triggering based on touched files
[scott] Rename setPostCommit to setAutoJob
[amyrvold] [BEAM-4645] Cleanup docker images on failure
[amaliujia] replace byte[] with String as Map key.
[scott] Fix ErrorProne warnings in sdks-java-core project
[scott] Enable ErrorProne analysis for all Java projects.
[scott] Update Findbugs annotations dependency to version with
[scott] Enable failOnWarnings for all projects.
[pablo] Removing debug string from UserScore.java
[scott] Add checkstyle validation to ban usage of @SuppressFBWarnings.
[amyrvold] [BEAM-3261] Fix pipeline failures by waiting for logging server to be
[github] Revert "improvements to fanout"
[klk] Apply rename of JobBuilder to Samza VR job
[xiliu] [BEAM-4641] Samza runner postcommit status in PR template
[kenn] Add bundle-ready SQL shell using application plugin
[relax] Use ByteBuddy to generate a coder class for a specific Schema.
[relax] Subclass proper generic type, removing need for extra typecast.
[relax] Address code-review comments.
[scott] Rename refactored Precommit source files to remove GradleBuild suffix.
[herohde] Run gofmt
[kenn] Reduce stdout spam from BeamModulePlugin
[scott] Update Pre-commit job names tracked in precommit_job_times.ipynb
[lcwik] [BEAM-4661] Define well known timer URN.
[herohde] [BEAM-4635] Avoid cross-compiling worker binary if linux-amd64
[klk] Increase website job timeout to 100 by default
[scott] Add Apache license headers
[ankurgoenka] Adding flinkMarterUrl as an option
[robertwb] Give the root transform in Python a non-empty name.
[herohde] [BEAM-4635] Do not remove worker binary on Go Dataflow staging
[amaliujia] Improve BigQuery write integration tests and support TIMESTAMP SQL type
[ryan.blake.williams] [BEAM-2810] use fastavro-blocks in Avro IO
[klk] Fix accidental license header javadoc comments
[klk] Fix HTML error in ApproximateQuantiles
[klk] Turn on spotless java everywhere
[aaltay] [BEAM-4606] Pin pytz version (#5751)
[scott] Update pre-commit filter to include migrated build rules
[klk] Do not email individuals about failures in Jenkins
[klk] Pipeline options in JDBC URI; default userAgent=BeamSQL
[klk] Spotless Java everywhere!
[klk] Simplify checkstyle now that spotless is in play
[klk] Temporarily disable JavadocParagraph and LineLength checkstyle
[scott] Remove blanket website from rat and add appropriate exclusions.
[github] improvements to fanout (#5800)
[herohde] Do not launch containers with transient semi_persistent_dir. Fails on
[herohde] Go-on-Flink: treat artifact staging EOF error as ok
[herohde] Go-on-Flink: make transform UniqueNames _globally_ unique
[herohde] Go-on-Flink: remove internal java runner class hack
[herohde] Go-on-Flink: fix swapped arguments in artifact staging
[herohde] Go-on-Flink: use clearer error if the artifact retrieval token or
[herohde] Go-on-Flink: fix Go to actually send worker_id gRPC header
[lukasz.gajowy] Remove mvn leftovers from jenkins IOIT job definitions
[klk] Rearrange checkArgument to help findbugs
[chamikara] Reduces the size of pretty string of BigQuery jobs
[mariagh] Add Python Nexmark suite
[herohde] CR: Don't enforce non-empty control worker_id yet
[xiliu] Make Samza runner in lexicographical order
[robertwb] [BEAM-4285] Extend side input handlers to handle multiple access
[relax] Rename RowType -> Schema and move to new schemas package.
[relax] Add classes for SchemaRegistry and SchemaProvider.
[relax] Add SchemaCoder, some more detail in SchemaProvider/Registery, and
[relax] Update to comment.
[relax] Integrate Schemas into PCollection Coder inference.
[relax] Add setSchema convenience method.
[relax] Fix compilation errors.
[relax] Refactor Schema: Schema now is a fully-specified based on primitive
[relax] Implement SchemaCoder and fix up schema classes.
[relax] Flesh out FieldAccessDescriptor.
[relax] Start plumbing through DoFn.
[robertwb] Rename several MultimapSideInput* classes to more generic SideInput*.
[relax] Start editing signatures
[relax] Finish plumbing schemas through.
[relax] Add test for schema PCollection.
[relax] Add withSchema to Create.java
[relax] Plumb schemas through OutputReceiver.
[relax] Add an implementation of SchemaProvider.
[relax] Add SchemaRegistry and a test for schema inference. Change schema
[relax] Add unit test for SchemaRegistry.
[relax] Add FieldAccessDescriptorTest
[relax] Add test for Create changes.
[relax] Plumb FieldAccess through.
[relax] Make sure everything is marked experimental with Kind.SCHEMAS
[relax] Add graph-construction time verification of schemas.
[relax] Fix build and style failures.
[relax] Address code-review comments.
[relax] Fixup after merge.
[relax] Apply spotless autoformat.
[lcwik] [BEAM-4659] Add well known timer coder to Java SDK.
[kirpichov] Makes fuser materialize PCollections used as side inputs
[relax] Fix compilation error introduced by merge.
[ankurgoenka] Uploading job server jar when running on remote flink environment
[ankurgoenka] Passing files to stage on flink explicitly while creating the
[ankurgoenka] Spotless
[robertwb] Remove obsolete comment about multimap fitting into memory.
[relax] Fix CheckStyle errors.
[ankurgoenka] Job service additionally need google cloud dependency to use GCS
[relax] Fix Apex runner.
[relax] Fix more breakages.
[relax] run spotless.
[kirpichov] Makes Python PortableRunner usable via PipelineOptions
[amaliujia] Remove unused system.out.println
[ankurgoenka] TODO to add more file systems
[kirpichov] Brings back hack for embedded FnApiRunner
[robertwb] [BEAM-3883] Stage files in the portability runner.
[robertwb] Fix tests due to api change.
[kirpichov] Changes portableWordCount gradle task to be runnable at master
[ankurgoenka] Adding non standard side input urn to flink
[relax] Fix issues.
[relax] Fix Spark issue.
[kirpichov] Fixes spotless errors at master
[mariagh] Allow streaming mobile examples to read from either a topic or a
[relax] Address code-review comments and fix a failing test.
[relax] Fix Apex runner.
[echauchot] [BEAM-2850] Add Nexmark PostCommit runs for spark, flink and direct
[relax] Change ParDoSchemaTest to ValidatesRunner, and exclude unsupported
[relax] Make worker changes backwards compatible with DataflowRunner. Once
[mariagh] Address review comments
[amaliujia] Support BigQuery read from SQL.
[relax] Add another backwards-compatible method.
[ekirpichov] Makes FileIO.match watermark advance even without new files
[pablo] Test case for providing experiments to pipeline
[kirpichov] [BEAM-4689] Reverts change of SDF key type
[daniel.o.programmer] [BEAM-3708] Adding grouping table to Precombine step.
[daniel.o.programmer] [BEAM-3708] Simplifying precombine grouping tables.
[klk] Add integrationTest task for GCP IO module
[klk] PubsubIO: Always use GCP project from PipelineOptions for subscriptions
[klk] Use BagState in PubsubSignal since it doesn't use SetState methods
[klk] Replace println with logging
[klk] More logging when PubsubJsonIT fails
[klk] Add start signal to TestPubsubSignal
[klk] Use futures in PubsubJsonIT LIMIT test
[klk] Integration test of Pubsub public dataset
[klk] Use test PipelineOptions in PubsubJsonIT LIMIT test
[klk] Sickbay PubsubReadIT on Dataflow
[klk] Fix import order
[mariagh] Add query 1 & 2
[mariagh] Clean up
[mariagh] Cleaup
[mariagh] Fix linter errors
[mariagh] Address review commnets; Reuse topic for queries
[github] Return timeout value to 180
[relax] Fix build.
[relax] Add SchemaProviderRegistrar class.
[klk] Revert accidental clobber of MinimalWordCount.java
[relax] Exclude schema tests from Samza runner.
[klk] Update BigQueryAvroUtils to support BigQuery's NUMERIC type.
[relax] Add DefaultSchema and fix failures turned up by plugins.
[klk] Convert Beam Row to Avatica Row in BeamEnumerableCollector
[kenn] sqlline dep to 1.4.0
[kenn] Fix broken javadoc
[kenn] Remove more extraneous printlns from build
[kenn] Remove aliased tables in Nexmark SQL query 5
[lcwik] Revert "Enabling gradle build to receive extra buildscript repositories"
[lcwik] Don't enable plugins which require you to be online when run in
[kenn] fixup! Remove aliased tables in Nexmark SQL query 5
[lcwik] [BEAM-4126] Delete Maven build files.
[kenn] Simplify/fix windowing check in BeamSortRel
[robertwb] Fix global hot key fanout setting.
[scott] Increase concurrency in ValidateRunner execution
[lcwik] [BEAM-3971, BEAM-4284] Remove fromProto for Pipeline and PTransform
[robinyqiu] Support @RequiresStableInput on Dataflow runner in Java SDK
[kenn] Run SQL operator tests through JDBC as well
[kenn] Test retrieval of TIMESTAMP via JDBC
[kenn] Instantiate $SUM0 as a SUM operator
[ankurgoenka] Read worker harness image name from pipeline option for portable runners
[ankurgoenka] Spotless
[kenn] Instantiate "+" as DATETIME_PLUS
[ankurgoenka] Support only a single environment in SdkComponents
[ankurgoenka] Renaming createEnvironment to createOrGetDefaultEnvironment.
[lcwik] [BEAM-4716] Remove findbugs declarations in build.gradle files since it
[relax] Address review commemnts.
[ccy] [BEAM-4007] Fix TODO style in typehints.py
[relax] Add schema inference from POJO.
[relax] Add BYTES type to schema, and POJO inference.
[relax] Add automatic getter/setter inference from POJOS. * Recursive POJOs
[relax] Handle setter for ByteBuffer fields. Setter will be called with a byte[]
[relax] Split Row into separate subclass for RowWithStorage and RowWithGetters.
[relax] Introduce FieldValueGetterFactory and FieldValueSetterFactory.
[relax] Add support for nested, array, and map types.
[relax] Refactor getter-based SchemaProvider into a common base class.
[relax] Refactor getter/setter conversion utilities into a library. This cleans
[relax] Refactor Schema inference from POJOs so much of the code can be reused
[relax] Add encoding byte BYTES type.
[relax] Make FieldValueGetter and FieldValueSetter generic types again.
[relax] Start adding JavaBean getters and setters.
[relax] Make sure that field order is stable, and make tests robust to
[relax] Simplify convertArray.
[amaliujia] support decimal type in reading from PubSub JSON
[ankurgoenka] Fix artifact retrival file system
[jiangkai] [BEAM-4723] Support date type plus interval
[ankurgoenka] Create SDKComponents with pipeline options
[echauchot] [BEAM-4707] Add timestamp field to Nexmark tables and use explicit mode
[lukasz.gajowy] [BEAM-4137] Divide separate PipelineOptions for elasticsearch tests
[lukasz.gajowy] [BEAM-4137] Remove unused options from IOTestPipelineOptions
[katarzyna.kucharczyk] Added integration test configuration to beam-examples gradle. Fixed
[markliu] Move gradle-spotless plugin to buildscript
[rober] Add a simple benchmark to the ParDo execution node.
[robertwb] Use main output for cold keys.
[pablo] Unpin GCP test requirement
[lcwik] [BEAM-4654] Translate timers as PCollections within the Java SDK.
[robertwb] Use local GCP credentials, if any, in local docker environment.
[cclauss] [BEAM-1251] Modernize Python 2 code to get ready for Python 3
[robertwb] Split bounded sources in larger than 1k chunks.
[cclauss] [BEAM-1251] Upgrade from buffer to memoryview (again)
[thw] Use the beam:option:value:v1 as the portable pipeline options
[thw] [BEAM-4733] Pass pipeline options from Python portable runner to job
[matthias] removed unnecessary install_aliases, import io and unicode imports
[thw] Fix Flink portable streaming translation executable stage output
[echauchot] [BEAM-4707] insert timestamp to bigQuery as seconds as expected by
[echauchot] [BEAM-4671] Skip MetricsPusherTest validates runner test for runners
[matthias] Futurize io subpackage
[matthias] incorporated all feedback for futurize io subpackage
[robertwb] object() takes no parameters
[amaliujia] remove decimal support in BigQuery SQL read.
[iemejia] [BEAM-4016] Invoke Setup and TearDown on SplitRestrictionFn and
[ankurgoenka] Initialization BeamFileSystem in FlinkExecutableStageFunction
[ankurgoenka] [BEAM-4644] Use SLF4J in ExecutableStageDoFnOperator
[ccy] Revert #5878 to unbreak Python postcommit
[ankurgoenka] Pull the docker image before starting docker container.
[iemejia] Rename COMBINE_TRANSFORM_URN to the more proper
[ccy] [BEAM-4003] Fix missing iteritems import
[relax] Address code-review comments.
[relax] Address code-review comments.
[github] Avoid silent error when calling unimplemented function.
[github] Avoid silent error when calling unimplemented function.
[relax] Add better validation to isSetter isGetter
[ekirpichov] Infer boundedness for SDF application at application time
[ekirpichov] [BEAM-4745] Revert "[BEAM-4016] Invoke Setup and TearDown on
[markliu] Remove :buildSrc from settings.gradle since it's not a subproject
[pablo] Logging relies on StateSampler for context
[ccy] [BEAM-4594] Beam Python state and timers user-facing API
[ccy] [BEAM-4593] Remove refcounts from the Python SDK
[ankurgoenka] FileSystems should throw exception if filesystem for schema is not found
[ankurgoenka] Make schema pattern stricter
[ccy] [BEAM-1251] Revert #5887 to unbreak Python PostCommit
[pablo] Improving signing of all published artifacts and adding test publication
[ccy] [BEAM-1251] Upgrade snappy and use a memoryview
[pablo] Removing extra publication
[cclauss] [BEAM-3761] Define cmp() in Python 3
[cclauss] [BEAM-3959] Add Python 3 undefined names to flake8
[amyrvold] [BEAM-3457] Upgrade version of gogradle and add examples and containers
[apilloud] [SQL] Inject JDBC rules through Hook.
[lcwik] [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
[apilloud] [SQL] Plumb pipelineOptions through IOSinkRel
[lcwik] [BEAM-4653] Add support to the Java SDK harness to execute timers.
[apilloud] [SQL] Default timezone is UTC
[lcwik] [BEAM-2915] Add support for handling bag user state to the
[amaliujia] add testArithmeticOperator
[altay] Wait on connection channel to be ready
[ekirpichov] Bumps Dataflow worker container version
[ekirpichov] [BEAM-4745, BEAM-4016] Reintroduces Setup and TearDown on
[amaliujia] simplify unbounded limit
[iemejia] [BEAM-4020] Add an HBaseIO read implementation based on SDF
[iemejia] [BEAM-4020] Add HBaseIO.readAll() based on SDF
[lukasz.gajowy] [BEAM-4691] Rename common_job_properties
[aromanenko.dev] [BEAM-4622] Makes required to call Beam SQL expressions validation
[aromanenko.dev] Check number of arguments at first
[lcwik] [BEAM-4764] Fix missing reference to project to be able to refer to the
[lcwik] [BEAM-4764] Fix java precommit to ensure that any checks in model/* are
[apilloud] [SQL] Apply JDBC rules globally
[rober] Add a small benchmark for metrics
[ankurgoenka] Warn on docker image pull fail
[robinyqiu] Fix bug in java mobile gaming example
[lcwik] [BEAM-4744] Enable parallel publishing/release
[qinyeli] Interactive runner and corresponding tests
[github] [BEAM-4648] Remove experimental Python RPC DirectRunner (#5777)
[s.lebedev] [BEAM-1251] Replace NameError-driven dispatch with ``past``
[altay] Install fastavro only in linux
[pablo] [BEAM-4718]Run gradle build before publish
[pablo] Add CODEOWNERS file (#5909)
[pablo] Full tracking of Batch side inputs. Performance tests show no regression
[kirpichov] Adds license header to CODEOWNERS
[kirpichov] Adds a naive implementation of bounded SDFs
[echauchot] Move echauchot from runner/core to runner/core/metrics in CODEOWNERS
[github] [BEAM-4752] Add support for newer dill dependency (#5931)
[elliottb] Fix the expected encoding of BigQuery's NUMERIC type when reading from
[ankurgoenka] Adding tet cases for matchResouces
[kirpichov] Makes SplittableDoFnTest exercise both bounded and unbounded SDFs.
[github] [BEAM-4752] Add dill compatibility for older versions of dill (#5949)
[ankurgoenka] Ignore generated vendor files in python container build
[kirpichov] Supports bounded SDF in all runners.
[relax] Add new Convert transform for schemas.
[relax] Fix two bugs: * proper handling of nullable fields. * Row stores
[relax] Add tests.
[relax] Fix broken test.
[relax] Add @Nullable annotation.
[github] [BEAM-4432] Adding Sources to produce Synthetic output for Batch
[ryan.blake.williams] send PutArtifactResponse in BeamFileSystemArtifactStagingService
[apilloud] [BEAM-4774] Add Nexmark SQL to postcommits
[aaltay] [BEAM-2810] fastavro integration test (#5862)
[kirpichov] Improve error messages on Spark and Flink SDF translation
[qinyeli] Interactive Beam -- yields Read Trans after apply()
[apilloud] [SQL] Actually run JDBC Jar Test
[apilloud] [SQL] Wrap PipelineOptions with correct class loader
[kirpichov] Converts BoundedReadFromUnboundedSource to a DoFn
[boyuanz] Automate release branch cut process
[yifanzou] [BEAM-4801] add dependency owners file
[lcwik] [BEAM-4769] Fix issue when performing coder inference for wildcard
[lcwik] [BEAM-4744] Fix runners/google-cloud-dataflow-java/examples-streaming
[ankurgoenka] Fixing Test case for beam Artifact Staging Service
[boyuanz] Fix javadoc broken
[daniel.o.programmer] [BEAM-3711] Enabling combiner lifting in Dataflow Runner.
[ankurgoenka] Fix for Flaky test BeamFileSystemArtifactServicesTest
[lcwik] [BEAM-4418] Fix duplicate source set declaration conflict.
[pablo] Add coders microbenchmark. (#5565)
[melissapa] [BEAM-3769] Fix broken JIRA links in README
[pablo] Moving to 2.7.0-SNAPSHOT on master branch.
[thw] [BEAM-4791] Portable Flink runner integration test.
[thw] Use shadowTest configuration since we don't plan to shade away the SDK
[thw] [BEAM-4791] Move executor to Before/After and use
[github] Increase the complete time out
[github] Minor javadoc fix up.
[amaliujia] BEAM-4820: Add timeout to Future.get when waiting for Pubsub messages
[github] Adding @angoenka to CODEOWNERS
[boyuanz] Fix javadoc build broken in synthetic for release branch
[garrettjonesgoogle] Bumping google cloud versions
[thw] Add JIRA reference.
[lcwik] Address PR comments.
[pablo] Fix bugs in 'cut release branch step'
[lcwik] Address PR comments.
[amaliujia] [BEAM-4806] improve unit tests for SqlLineMain
[qinyeli] Added a round trip through proto in Interactive runner.
[github] Add version bound to numpy.
[echauchot] CodeOwners: Narrow down extensions-java to metrics and assign echauchot
[jxlewis] Added ClientConfiguration to AwsOptions and S3FileSystem
[jxlewis] Updated documentation, added BEAM-4828 to In-Progress
[github] Fix indentation.
[github] SQS -> Amazon SQS
[boyuanz] Added '--continue' switches into nightly build
[supercclank] Set BatchPath on google.api.services.storage.Storage Storage Builder
[kirpichov] Converts SolrIO away from BoundedSource
[supercclank] Change to using java.nio.file.Paths over string concatenation for batch
[apilloud] [BEAM-4761] Run Nexmark on Dataflow
[github] [BEAM-4832] Fix concurrent race in datamgr.go
[github] Update datamgr.go
[carlos.alonso] Adds the BigQueryInsertError model class
[carlos.alonso] Adds BigQueryInsertErrorCoder and tests
[carlos.alonso] Adds extended errors info to WriteResult
[carlos.alonso] Makes streaming methods generic but limited to the implemented error
[carlos.alonso] Removes ErrorContainers auxiliary class. Adds BigQueryServicesImpl test
[carlos.alonso] Makes BigQueryIO.Write to be configurable to decide which kind of errors
[carlos.alonso] Removes unneeded coder registration
[carlos.alonso] Simplifies BigQueryInsertError codification
[carlos.alonso] A few style improvements
[katarzyna.kucharczyk] Changed order of installing perfkit requirements and beam requirements
[thw] Remove dead link from README.
[amyrvold] [BEAM-4831] Ignore failures during :beam-sdks-go:vet to allow ./gradlew
[aaltay] Add automation script for verifying release build (#5973)
[ekirpichov] Avoids use of discouraged Duration(long) ctor
[qinyeli] Interactive Runner -- keep_alive
[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.1
[daniel.o.programmer] [BEAM-3709] Implementing new combine URNs in python.
[aaltay] Add bash script to automate "Preparation for GPG" (#6015)
[github] Fixing log message
[lukasz.gajowy] [BEAM-4845] Make BigQueryServices and FakeDatasetService public to fix
[amyrvold] Fix [BEAM-4847] by manually setting max workers and jvm memory and add
[relax] Enable schemas for Apex runner.
[relax] Enable Schemas on samza runner.
[relax] Enable schemas for Flink runner.
[relax] Enable schemas for Spark.
[relax] Fix bugs.
[relax] Fix Apex breakage.
[ehudm] Remove CODEOWNERs.
[kedin] [SQL] Enable running BeamSqlLine from gradle
[thw] [BEAM-4847] Reduce Gradle JVM Xmx to 4g to fix Jenkins build failures.
[aaltay] Automate 'Start a snapshot build' step in beam release guide (#6042)
[lukasz.gajowy] [BEAM-4838] Add dockerfile for standalone Jenkins. Plugins included.
[dariusz.aniszewski] separate virtualenvs for Beam and PerfKit
[lcwik] [BEAM-4866] Fix missing licenses.
[pablo] Removing scoped metrics container
[pablo] Remove old style metrics context management
[garrettjonesgoogle] Bumping versions that were missed in #5988
[altay] Remove reference to dataflow-distribution.properties
[lcwik] [BEAM-4176] Initial implementation for running portable runner tests
[pablo] Fix Java Nightly Snapshot Failures
[lcwik] [BEAM-4629] Output the names of the failing licenses as part of the
[aaltay] [BEAM-4859] Enable Python VR tests in streaming in postcommit task
------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f aa2b585a5356ce3b607dd3d17a3c6ab2ce65353b
Commit message: "Merge pull request #6056: [BEAM-4836] separate virtualenvs for Beam and PerfKit"
> git rev-list --no-walk 0892e23940a584a170e2668d86b7bdc50c03e024 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266031542697616565.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2118418907451248094.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1013385759992617214.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1440131868588494871.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3203404953492079456.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3388230821306939726.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins8681682376553589136.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1014785923945432727.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7488451273964902903.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7216896748234028238.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
Traceback (most recent call last):
File "src/sdks/python/setup.py", line 46, in <module>
PACKAGE_VERSION = get_version()
File "src/sdks/python/setup.py", line 41, in get_version
exec(open(os.path.normpath('./apache_beam/version.py')).read(), global_names) # pylint: disable=exec-used
IOError: [Errno 2] No such file or directory: 'apache_beam/version.py'
Build step 'Execute shell' marked build as failure
Build failed in Jenkins: beam_PerformanceTests_Python #1254
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PerformanceTests_Python/1254/display/redirect>
------------------------------------------
[...truncated 1.72 KB...]
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins2255626873956628632.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9165999473588564426.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins9063788179791268742.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5884993017406052429.sh
+ virtualenv .beam_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1342281336343959987.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python2>
Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.perfkit_env/bin/python>
Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3118154934927439155.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins6504331171475556563.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins4209305034972557352.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins5351899519785124003.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in /usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in /usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (3.2.0)
Requirement already satisfied: future<1.0.0,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.16.0)
Requirement already satisfied: fastavro==0.19.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.19.7)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.5.20)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.90.4)
Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (7.0.1)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.26.0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: nose>=1.3.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.3.7)
Collecting numpy<2,>=1.14.3 (from apache-beam==2.7.0.dev0)
Using cached https://files.pythonhosted.org/packages/85/51/ba4564ded90e093dbb6adfc3e21f99ae953d9ad56477e1b0d4a93bacf7d3/numpy-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) (1.9.0)
Requirement already satisfied: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio<2,>=1.8->apache-beam==2.7.0.dev0) (1.1.6)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.18.4)
Requirement already satisfied: pbr>=0.11 in /usr/lib/python2.7/dist-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.8.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.7.0.dev0) (1.0.2)
Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python2.7/dist-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.4.2)
Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (0.2.2)
Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.7.0.dev0) (3.4.2)
Requirement already satisfied: setuptools in ./.beam_env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.7.0.dev0) (40.0.0)
Requirement already satisfied: pyparsing>=2.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from pydot<1.3,>=1.2.0->apache-beam==2.7.0.dev0) (2.2.0)
Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (0.14.1)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.7.0.dev0) (1.5.3)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.4)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.25.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.7.0.dev0) (1.22)
Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.7.0.dev0) (1.5)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.15.16)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.11.4)
Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (0.0.3)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (1.5.0)
Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (3.8)
Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.7.0.dev0) (2.1.0)
Installing collected packages: numpy, apache-beam
Found existing installation: numpy 1.13.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'numpy'. No files were found to uninstall.
Found existing installation: apache-beam 2.7.0.dev0
Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Compressed_TextIOIT_HDFS/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env>
Can't uninstall 'apache-beam'. No files were found to uninstall.
Running setup.py develop for apache-beam
Successfully installed apache-beam numpy-1.15.0
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins1563680810598338892.sh
+ .beam_env/bin/python src/sdks/python/setup.py sdist --dist-dir=src/sdks/python/target
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
running sdist
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
running egg_info
creating apache_beam.egg-info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md
running check
creating apache-beam-2.7.0.dev0
creating apache-beam-2.7.0.dev0/apache_beam.egg-info
copying files to apache-beam-2.7.0.dev0...
copying apache_beam.egg-info/PKG-INFO -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/SOURCES.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/dependency_links.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/entry_points.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/not-zip-safe -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/requires.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
copying apache_beam.egg-info/top_level.txt -> apache-beam-2.7.0.dev0/apache_beam.egg-info
Writing apache-beam-2.7.0.dev0/setup.cfg
creating src/sdks/python/target
Creating tar archive
removing 'apache-beam-2.7.0.dev0' (and everything under it)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins3779598970039815852.sh
+ .perfkit_env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) (0.2.2)
Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) (2.10)
Requirement already satisfied: setuptools in ./.perfkit_env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) (40.0.0)
Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) (2.6.0)
Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) (1.4)
Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) (3.2.0)
Requirement already satisfied: PyYAML==3.12 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) (3.12)
Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) (0.8.1)
Requirement already satisfied: numpy==1.13.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) (1.13.3)
Requirement already satisfied: functools32 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) (3.2.3.post2)
Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) (0.5.5)
Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) (0.3.0)
Requirement already satisfied: timeout-decorator in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 26)) (0.4.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) (1.0)
Requirement already satisfied: colorama; extra == "windows" in /home/jenkins/.local/lib/python2.7/site-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) (0.3.9)
Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.0)
Requirement already satisfied: requests>=2.9.1 in /usr/local/lib/python2.7/dist-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18.4)
Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.11.0)
Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python2.7/dist-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.2.2)
Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2018.4.16)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.22)
Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.1.6)
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (1.0.22)
Requirement already satisfied: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) (2.18)
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins7173993871705149011.sh
+ .perfkit_env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src> --official=true --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-07-26 10:46:45,580 b302cf0c MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/pkb.log>
2018-07-26 10:46:45,580 b302cf0c MainThread INFO PerfKitBenchmarker version: v1.12.0-729-ga00bd87
2018-07-26 10:46:45,581 b302cf0c MainThread INFO Flag values:
--beam_sdk=python
--k8s_get_wait_interval=10
--python_binary=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python>
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--beam_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src>
--k8s_get_retry_count=36
--benchmarks=beam_integration_benchmark
2018-07-26 10:46:45,875 b302cf0c MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo.
2018-07-26 10:46:46,063 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark
2018-07-26 10:46:46,067 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark
2018-07-26 10:46:46,068 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/gradlew> clean assemble --stacktrace --info -DintegrationTestRunner=dataflow
2018-07-26 10:52:45,185 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark
2018-07-26 10:52:45,278 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz>
2018-07-26 11:02:45,286 b302cf0c Thread-3 ERROR IssueCommand timed out after 600 seconds. Killing command "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz".>
2018-07-26 11:02:45,296 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Ran: {<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/bin/python> setup.py nosetests --tests=apache_beam.examples.wordcount_it_test --attr=IT --test-pipeline-options="--project=apache-beam-testing" "--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it" "--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it" "--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output" --runner=TestDataflowRunner --sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/target/apache-beam-2.7.0.dev0.tar.gz}> ReturnCode:-9
STDOUT: running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
STDERR: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py>:398: UserWarning: Normalizing '2.7.0.dev' to '2.7.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/gen_protos.py>:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
2018-07-26 11:02:45,297 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-26 11:02:45,298 b302cf0c MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark
2018-07-26 11:02:45,301 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark
Traceback (most recent call last):
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 804, in RunBenchmarkTask
RunBenchmark(spec, collector)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 668, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 548, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py",> line 159, in Run
job_type=job_type)
File "<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py",> line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-07-26 11:02:45,301 b302cf0c MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue.
2018-07-26 11:02:45,317 b302cf0c MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-07-26 11:02:45,318 b302cf0c MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/pkb.log>
2018-07-26 11:02:45,318 b302cf0c MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/b302cf0c/completion_statuses.json>
Build step 'Execute shell' marked build as failure